-- Logs begin at Mon 2022-09-26 04:47:42 UTC, end at Mon 2022-09-26 05:47:40 UTC. -- Sep 26 05:14:30.417449 dv-cinder-859186-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 26 05:14:30.425279 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 26 05:14:30 2022] *** Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: nodename: dv-cinder-859186-2-iscsi Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: machine: x86_64 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: clock source: unix Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: pcre jit disabled Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: detected number of CPU cores: 4 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: current working directory: / Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: detected binary path: /usr/bin/uwsgi-core Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: your processes number limit is 47680 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: your memory page size is 4096 bytes Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: detected max file descriptor number: 2048 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: lock engine: pthread robust mutexes Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: thunder lock: enabled Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 26 05:14:30.428777 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 26 05:14:30.466851 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: Python main interpreter initialized at 0x5562b0cd49c0 Sep 26 05:14:30.466851 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: python threads support enabled Sep 26 05:14:30.466851 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: your server socket listen backlog is limited to 100 connections Sep 26 05:14:30.466851 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: your mercy for graceful operations on workers is 90 seconds Sep 26 05:14:30.467233 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: mapped 403077 bytes (393 KB) for 2 cores Sep 26 05:14:30.467314 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: *** Operational MODE: preforking *** Sep 26 05:14:30.467551 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: *** uWSGI is running in multiple interpreter mode *** Sep 26 05:14:30.467551 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: spawned uWSGI master process (pid: 81527) Sep 26 05:14:30.467761 dv-cinder-859186-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 26 05:14:30.468900 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: spawned uWSGI worker 1 (pid: 81542, cores: 1) Sep 26 05:14:30.468900 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: spawned uWSGI worker 2 (pid: 81543, cores: 1) Sep 26 05:14:30.468900 dv-cinder-859186-2-iscsi devstack@keystone.service[81527]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 26 05:14:34.805690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:14:34.805690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: from cryptography.utils import int_from_bytes Sep 26 05:14:34.806496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:14:34.806496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: from cryptography.utils import int_from_bytes Sep 26 05:14:34.806914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:14:34.806914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: from cryptography.utils import int_from_bytes Sep 26 05:14:34.807024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:14:34.807024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: from cryptography.utils import int_from_bytes Sep 26 05:14:34.855598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 05:14:34.855784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 05:14:34.855962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] command line args: [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 05:14:34.856054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 05:14:34.856189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 05:14:34.856287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 05:14:34.856410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] command line args: [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 05:14:34.856489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ================================================================================ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 05:14:34.856664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] admin_token = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.856748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 05:14:34.856858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ================================================================================ {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 05:14:34.856942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_dir = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] admin_token = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] config_source = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] config_dir = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] debug = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] config_source = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.857899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] debug = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.858966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_config_append = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_config_append = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859485 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_dir = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_dir = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859861 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.859931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_file = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.860973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.861969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.862708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.862830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.862942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.863836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] publish_errors = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] publish_errors = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.864982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] standard_threads = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.865942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] standard_threads = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_journal = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_json = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.866864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_stderr = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] use_syslog = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] use_journal = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] use_json = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] use_stderr = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.867840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.867952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] use_syslog = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.868063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:14:34.868321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.868906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.869968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.870988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.871908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.872921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.873928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.874967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.875987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.876928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.877987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.878970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.879951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.880935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.881896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.882925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.883990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.884971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.885991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.886913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.887921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.888984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.889967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.890904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.891903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.892937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.external = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.893991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.password = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] auth.token = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.external = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.894918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.password = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] auth.token = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.895952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.896900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.897942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.898892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.899941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.900916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.901962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.902883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.903895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.904956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.905984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.906997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.907929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.908788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.909406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.909584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.909706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.910943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.911979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.912912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.913934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914398 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.914976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.915901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.916886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.917931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.918965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.919892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.920951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.921993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.922917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.923939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.924060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.924176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.924315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.924431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.924568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.925973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.926913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.927911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] role.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.928981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] role.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.929904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.930947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.931994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.932937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.933971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.934885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.935923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.936982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.937904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.938952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.939980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.940989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.connection = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.941984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.942894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.943803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.944762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:14:34.945090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 05:14:34.973632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.973839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.974072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.976996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.977231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.977452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.982611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:34.985766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.012119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.012376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.013368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.013601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.066707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.066924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.116100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.116360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.116477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.116719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.271924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.272932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.273063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.273149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.273270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.295787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.296932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.297014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.297139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.297217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.297347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.308775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.309087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.321960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.322871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.323634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:14:35.340038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.340163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.340278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:14:35.340363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:14:35.340606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.340708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.340786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:14:35.340860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:14:35.341062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.341919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.351450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.351545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.351727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.351812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.351997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.352094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.352304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.352391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.359286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.359373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.362726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.362895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.363035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.363850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.364975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.365151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.365233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.365415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.365497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.365686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.389001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.389498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.390453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.390736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.390971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.391262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.395504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.395785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.396016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.396103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.396331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.396621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.402474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.402759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.403008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.403301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.408319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.408623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.408949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.409235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.409881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.410919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.426241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.426511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.427067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.427351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.430320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.431193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.433513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.433810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.434424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.434727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.437968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.438912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.441650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.442915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.444835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.445135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.445420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.445706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.445981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.446134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.446436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.446725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.447012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.447296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.447582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.461906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.462195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.462461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.462721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.463651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.463944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.464216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.464509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.469547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.471073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.471893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.473250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.473483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.473567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.475009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.475300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.476177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.476500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.476788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.477070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.477452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.478193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.478491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.478779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.479065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.479464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.489354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.489637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.489917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.490196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.490574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.490853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.491130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.491499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.491789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.492076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.492393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.492785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.493082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.493376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.506922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.509419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.509842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.512371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.512735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.514196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.515275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.515636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.516082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.516330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.516817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.518314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.518600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.519012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.522818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.524273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.525456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.526913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.527162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.527555 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.527839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.528114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.529842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.530246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.530540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.530829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.536519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.536808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.537086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.539341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.539644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.539932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.543917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.544202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.544516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.544799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.545071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.545341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.545611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.545892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.546171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.546436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.546706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.546885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:14:35.546975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.547177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.547266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.547466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.547557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.547749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.548029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.548344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.548628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.548912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.549184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.549475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.550404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.550703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.550975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.551266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:14:35.581890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5562b0cd49c0 pid: 81542 (default app) Sep 26 05:14:35.585086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71dd1d-224c-4899-a468-427b371171f7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:35.585369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71dd1d-224c-4899-a468-427b371171f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:35.585541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5562b0cd49c0 pid: 81543 (default app) Sep 26 05:14:35.585638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71dd1d-224c-4899-a468-427b371171f7 None None] PATH_INFO: `/v3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:35.589734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Mon Sep 26 05:14:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:42.363778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:42.364296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:42.364296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:42.368452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:42.373820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:42.374215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:42.374351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:42.445117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: INFO dbcounter [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] Registered counter for database keystone Sep 26 05:14:42.464444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG oslo_db.sqlalchemy.engines [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 05:14:42.470678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writer thread running {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 05:14:42.649010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG passlib.handlers.bcrypt [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 05:14:42.649316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG passlib.handlers.bcrypt [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 05:14:42.707131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:42.708965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:42.837720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 05:14:43.322272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:43.391679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: INFO dbcounter [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] Registered counter for database keystone Sep 26 05:14:43.413641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG oslo_db.sqlalchemy.engines [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 05:14:43.419720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writer thread running {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 05:14:43.539618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae04956-a7cc-4f1d-bcfb-9bee4d7b2ae8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:43.543820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:43.544467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:43.544748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:43.545016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:43.553067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:43.604988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING keystone.common.rbac_enforcer.enforcer [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 05:14:43.605326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8ff3d-215b-4c05-8658-31c98649d78c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:43.614422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Sep 26 05:14:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:14:43.620636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:43.637409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9d5d-d78a-4c83-b4f7-e0625a3fffd4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:43.641790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:43.642847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:43.643127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:43.644022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] PATH_INFO: `/v3/projects/3af173ac0b7d40069de1fa09770699a3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:43.645713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] RBAC: Authorizing `identity:get_project(project_id=3af173ac0b7d40069de1fa09770699a3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:43.698839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 05:14:43.699177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f9eb2-45c9-4503-aeba-02ec1e015c7d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:43.703612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Mon Sep 26 05:14:43 2022] GET /identity/v3/projects/3af173ac0b7d40069de1fa09770699a3 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:45.809262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8657133-2cd7-4294-83d2-f460244ae7b6 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:45.809855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8657133-2cd7-4294-83d2-f460244ae7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:45.809855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8657133-2cd7-4294-83d2-f460244ae7b6 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:45.813858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:45.819855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:45.820309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:45.820492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:45.842141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:45.843691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:45.852290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 05:14:46.123169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8657133-2cd7-4294-83d2-f460244ae7b6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:46.135116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8657133-2cd7-4294-83d2-f460244ae7b6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:46.139300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:46.139883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:46.140158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:46.140461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:46.148083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:46.149929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b43a5-8b0b-46de-9abb-28e9353272ce admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:46.158381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Sep 26 05:14:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:14:46.164072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:46.173095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64a2cb-4afa-4aa0-a307-ea3c95e99ff2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:46.178035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:46.178647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:46.178920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:46.179189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:46.180514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:46.182314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1da9-f372-4fd3-81f4-835ed49cf437 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:46.183812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Mon Sep 26 05:14:46 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:48.416858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef77620-1a7d-4244-80f4-48e5df18c1a7 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:48.417344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef77620-1a7d-4244-80f4-48e5df18c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:48.417344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef77620-1a7d-4244-80f4-48e5df18c1a7 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:48.418615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:14:48.423995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:48.424409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:48.425223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:48.436237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f610-220d-4616-8b8f-72200ec2fc9e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:48.436541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f610-220d-4616-8b8f-72200ec2fc9e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:48.436813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f610-220d-4616-8b8f-72200ec2fc9e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:48.438320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:14:48.443257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:48.443446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:48.443624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:48.443788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:48.444661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:48.490640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG passlib.handlers.bcrypt [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81543) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 05:14:48.491020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG passlib.handlers.bcrypt [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81543) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 05:14:48.525251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:48.540231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:48.541836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:48.641759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:14:48.795527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:48.819896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:48.898903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5924554-bc87-4a45-a46a-33a5bdcef2a6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:48.909533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:48.932505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:48.933153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:48.933432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:48.933707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:48.945266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:14:48.954052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:14:48.955196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:14:48.956216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:14:48.963082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.013342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a42335-1106-4bc0-bd38-c2c69db73ffd None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.015544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.043177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.053204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.054004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.054284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.054559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.062961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87902531-7122-4a4d-9f43-ef94bfe5b4ca None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.096800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.097634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.097915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.098189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.098854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:49.099154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.100580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.112835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:49.118585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.125196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.126714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.217804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658797-e904-4439-8d28-d7f5969b599e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.222898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:14:49.228581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.231240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.253427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.254057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.254332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.254602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.255731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.257210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.258762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:14:49 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:49.264230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.278341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.326269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7c70a-f652-464e-a285-4dbeaa3725ca None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.359467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.360353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.360635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.360918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.366189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ceaa-e5a8-4d1d-8c4f-ede6c875631b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.379241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.380760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:14:49.399209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:14:49.400082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:14:49.401074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:14:49.401074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:14:49.401074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:14:49.401074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:14:49.401287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.401531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.401646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.401916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.403039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.404505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.405978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.406353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:49.412578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:49.461549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.492467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:49.509014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d0a86-fbff-414f-b9ee-fb982bf0978f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.511195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf51cd0-368e-4f66-a492-37c2f21b65d4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:49.542485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.543090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.543371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.543639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.544302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:49.544600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.544696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:49.545273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:49.545548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:49.545819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:49.545990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d085805-0fbf-4a7c-adf9-fb0e12c72d26 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.546525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.domain.id=default, user.id=6c403c92b6314d26999d2aed9104bf3f) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:49.546827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:49.548243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e1412-b3be-4052-bef0-8cc19b0e73a9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:49.554701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Sep 26 05:14:49 2022] GET /identity/v3/role_assignments?user.id=6c403c92b6314d26999d2aed9104bf3f&scope.domain.id=default&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:49.563524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:52.038071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0d9b-338d-4d73-83c0-708018815d81 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.038623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0d9b-338d-4d73-83c0-708018815d81 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.038623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0d9b-338d-4d73-83c0-708018815d81 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.040164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:52.046055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab178-468e-4da0-b405-f518d46fe75f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.046996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab178-468e-4da0-b405-f518d46fe75f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.047510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab178-468e-4da0-b405-f518d46fe75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.068540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ea8ab178-468e-4da0-b405-f518d46fe75f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:52.070127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ea8ab178-468e-4da0-b405-f518d46fe75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:52.148784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:52.226856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4d6d-7f90-403d-ae6f-3dc4e3b80334 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.227182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4d6d-7f90-403d-ae6f-3dc4e3b80334 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.227465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4d6d-7f90-403d-ae6f-3dc4e3b80334 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.229122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:52.243995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7c2a-dbab-430f-b806-1d2e976faab1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.244358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7c2a-dbab-430f-b806-1d2e976faab1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.244923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb7c2a-dbab-430f-b806-1d2e976faab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.264113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b6eb7c2a-dbab-430f-b806-1d2e976faab1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:52.265380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b6eb7c2a-dbab-430f-b806-1d2e976faab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:52.310986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:52.338763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 12/25] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:52.360306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62b4a9-93a9-46d8-bd2c-b5ff1609b12e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.360741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62b4a9-93a9-46d8-bd2c-b5ff1609b12e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.361142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62b4a9-93a9-46d8-bd2c-b5ff1609b12e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.363046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 14/26] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:52.368400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.368693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.368972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.396987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:52.398176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:52.488070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 13/27] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:14:52.553377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62b4a9-93a9-46d8-bd2c-b5ff1609b12e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:52.688184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62b4a9-93a9-46d8-bd2c-b5ff1609b12e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:52.728289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:52.728930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:52.729211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:52.729498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:52.741165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:52.742684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:52.746238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:14:52.761763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:14:52.763938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:14:52.765309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:14:52.765309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:14:52.765309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:14:52.765309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 15/28] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:14:52.769178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:52.771230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:52.933019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618789-25bc-4301-b7f7-7e2f4ff0556b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:52.975539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918772a5-52a0-408b-9ee6-abdcb71cb9e1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:52.990838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.000282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.000903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.001178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.001447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.022523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:14:53.028115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:14:53.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 16/29] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:14:53.032465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.052239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.052977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.053155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.053426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.054092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:53.054396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.055808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.076099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 14/30] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:53.087073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.125909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.175744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e081-88df-4e85-965b-cacc2fdaf485 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.214755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.215781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.216374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.216835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.217894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.219470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1f8f3-afc0-43dc-acd9-0588f1b5978a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.249979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726798cf-e978-4a94-9060-1b86e615f850 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.285330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.285951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.286230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.286502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.287164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:53.287467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.288900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.302809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 15/31] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:53.303739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.332889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.360406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 17/32] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 05:14:53 2022] POST /identity/v3/roles => generated 219 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:14:53.363105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38c61-8636-441c-b759-482b8dbf75d8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.363590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38c61-8636-441c-b759-482b8dbf75d8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.364059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38c61-8636-441c-b759-482b8dbf75d8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.365892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 18/33] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:53.366913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38c61-8636-441c-b759-482b8dbf75d8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.415656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ed948-b810-4d83-9467-482653bc4692 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.452709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.453326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.453605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.453880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.454993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.456293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.500354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38c61-8636-441c-b759-482b8dbf75d8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.524483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.548689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.550601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.551094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.551563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.553177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.555163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfe28b-fb79-4392-85c5-df42a2b3103b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.557100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 19/34] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:14:53 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:53.558473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.558959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.559481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.579807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:53.581939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:53.595362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 16/35] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:14:53 2022] POST /identity/v3/projects => generated 301 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:14:53.597044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.667031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 20/36] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:53.705411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.729365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9e5c1-dce2-48fb-87e7-cefd7e333ae0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.766332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.766910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.767193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.767467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.768836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.770646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeb796-1a17-4e43-a89f-d07324564b60 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:53.772296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 17/37] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:53.778349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:53.937772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86978e85-6ca1-4c7c-a8b8-27b042701092 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:53.955788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:53.977226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:53.977899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:53.978172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:53.978446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] PATH_INFO: `/v3/domains/default/users/6c403c92b6314d26999d2aed9104bf3f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:53.980420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6c403c92b6314d26999d2aed9104bf3f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:53.981868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea08ab-a2c0-45cf-beb3-cd6153b4c088 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:54.002146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 21/38] 10.222.0.49 () {58 vars in 1361 bytes} [Mon Sep 26 05:14:53 2022] PUT /identity/v3/domains/default/users/6c403c92b6314d26999d2aed9104bf3f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:14:54.010932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965899cc-4923-4e20-ae10-4cf982823929 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.011231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965899cc-4923-4e20-ae10-4cf982823929 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.011479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965899cc-4923-4e20-ae10-4cf982823929 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.014556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 18/39] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:54.020660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.020948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.021198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.047710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:54.049014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:54.054661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043d6b-65df-4477-8e2b-ae37ff3511ca None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.055148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043d6b-65df-4477-8e2b-ae37ff3511ca None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.055251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043d6b-65df-4477-8e2b-ae37ff3511ca None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.056695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 19/40] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:14:54.063690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.063970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.064244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.094178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:54.095782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:54.141329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 22/41] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:54.142198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:54.196596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 20/42] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:14:54.264734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b7f9-cb4c-4aca-b1e5-05fe02ae0c20 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:54.268897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:54.307531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:54.308070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.308396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.308726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.309402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:54.310779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:54.326920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 23/43] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 05:14:54 2022] POST /identity/v3/roles => generated 225 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:14:54.455196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:54.470913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:54.562992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:54.593568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fc848-6dc8-4bf3-8245-13138d57a250 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:54.632735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:54.633836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.634388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.634905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.635819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:54.637582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:54.654272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 21/44] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 05:14:54 2022] POST /identity/v3/roles => generated 223 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:14:54.696059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7aa4-dd09-489b-bcf0-5f83af379642 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:54.735428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:54.736443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.737452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.738047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.739456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:54.741166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3d6e5-87d9-41f7-b46a-613bde12d825 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:54.743083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 24/45] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:54.749110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:54.865702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e32eb-8c97-4e2a-bf5d-dfce5d755bc8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:54.900469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:54.901091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:54.901369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:54.901639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:54.902295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:54.903764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcf7f-b5ae-46e4-98e1-0ab411184955 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:54.913783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:54.931432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 22/46] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:14:54 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:14:56.284790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f426d-854f-4d59-8057-260474c1430a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:56.285330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f426d-854f-4d59-8057-260474c1430a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:56.285330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f426d-854f-4d59-8057-260474c1430a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:56.286657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 25/47] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:14:56.292049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:56.292507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:56.292651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:56.310829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:56.312007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:56.376830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 23/48] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:14:56.654194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f426d-854f-4d59-8057-260474c1430a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:56.684733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:56.755357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f426d-854f-4d59-8057-260474c1430a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:56.790813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:56.791446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:56.791722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:56.791990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:56.802705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:14:56.807726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:14:56.808766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:14:56.809518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 26/49] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:14:56.814777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:56.911844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3cf5a-b24c-40ee-a341-56dff7ab03fa None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:56.931380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:56.946447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:56.947109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:56.947390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:56.947662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:56.948352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:56.948654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:56.950030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:56.958009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 24/50] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:56.963847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:57.058167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:57.068605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d39b7-f6b2-4774-bcd6-f1ab49f871ba None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:57.101099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:57.101726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.101999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.102264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.103379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:57.104991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8188-7cee-40e0-bef7-720007cfbfa3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:57.106549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 27/51] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:14:56 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:14:57.112811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:57.163867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240434b-dcdd-443b-8e43-b0a7f1a4c105 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.164064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240434b-dcdd-443b-8e43-b0a7f1a4c105 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.164375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240434b-dcdd-443b-8e43-b0a7f1a4c105 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.165701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 28/52] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:14:57.173807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.174088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.174908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.193306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:14:57.194486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:14:57.213500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dae7a-abe8-492a-8cea-f4ed7149343a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:57.249451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:57.250070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.250348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.250621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.251756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:57.253269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:57.254827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 25/53] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:57.259434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 29/54] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:14:57.260539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:57.310219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:57.461312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d279069-ef6e-428d-90b2-221add02a6be None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:57.495025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:57.495622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.495900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.496170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.496959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.domain.id=default, user.id=6c403c92b6314d26999d2aed9104bf3f) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:14:57.497267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:57.498679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:57.507144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 26/55] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Sep 26 05:14:57 2022] GET /identity/v3/role_assignments?user.id=6c403c92b6314d26999d2aed9104bf3f&scope.domain.id=default&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 530 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:57.538452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:57.632879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:57.642913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c2d8-8d1e-43e1-8d13-1df344f0292d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:57.679315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:57.679949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.680223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.680526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.681616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:57.683099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7fb7d-23ae-43fd-aa62-f6bd7c3419c4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:57.684679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 30/56] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:14:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:14:57.690637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:14:57.790689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25732-45b2-421c-96b3-ef0a9d95f97a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:14:57.803765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:14:57.823647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:14:57.824278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:14:57.824557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:14:57.824828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:14:57.825488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:14:57.826911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c39d2-500a-49fc-8b1c-949c961718e0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:14:57.852743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 27/57] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Sep 26 05:14:57 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:00.031045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14aa1-2b4b-4874-a057-a7aaf684c734 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:00.031566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14aa1-2b4b-4874-a057-a7aaf684c734 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:00.031651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe14aa1-2b4b-4874-a057-a7aaf684c734 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:00.032965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:00.038771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483291-2889-4713-bd81-545631e4924d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:00.039199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483291-2889-4713-bd81-545631e4924d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:00.039317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483291-2889-4713-bd81-545631e4924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:00.057071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b8483291-2889-4713-bd81-545631e4924d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:00.058249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b8483291-2889-4713-bd81-545631e4924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:00.123357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:00.400399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe14aa1-2b4b-4874-a057-a7aaf684c734 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:00.499387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe14aa1-2b4b-4874-a057-a7aaf684c734 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:00.532717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:00.533333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:00.533653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:00.533921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:00.535072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:00.536541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36af1-69eb-4f00-8957-cd943217d3e3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:00.538064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:00.543501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483291-2889-4713-bd81-545631e4924d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:00.633529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:00.639687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483291-2889-4713-bd81-545631e4924d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:00.673975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:00.674549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:00.674824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:00.675092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:00.675731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:00.677182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b89ac9-2bd1-452a-8a92-915ddcce0577 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:00.702014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:00 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:03.975945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7fe2-8ca5-4683-bb9d-1c991bf84bd7 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:03.976815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7fe2-8ca5-4683-bb9d-1c991bf84bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:03.977232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7fe2-8ca5-4683-bb9d-1c991bf84bd7 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:03.978764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:03.984180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5818d-431c-46e9-ac01-9c751103d4d5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:03.984598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5818d-431c-46e9-ac01-9c751103d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:03.984769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5818d-431c-46e9-ac01-9c751103d4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.009766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-aec5818d-431c-46e9-ac01-9c751103d4d5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:04.010983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aec5818d-431c-46e9-ac01-9c751103d4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:04.065169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4492949-cbf7-4a4d-9814-c88db0d28113 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.065574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4492949-cbf7-4a4d-9814-c88db0d28113 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.065735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4492949-cbf7-4a4d-9814-c88db0d28113 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.067050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 34/63] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:04.072031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a3d7-3f00-4fc8-87a6-f5662802dc73 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.072348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a3d7-3f00-4fc8-87a6-f5662802dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.072618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a3d7-3f00-4fc8-87a6-f5662802dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.095931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0e92a3d7-3f00-4fc8-87a6-f5662802dc73 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:04.097684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0e92a3d7-3f00-4fc8-87a6-f5662802dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:04.130644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 30/64] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:04.140789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:04.222671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 35/65] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:04.338380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72477223-720e-4474-b867-3327b4b0591a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.338666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72477223-720e-4474-b867-3327b4b0591a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.338855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72477223-720e-4474-b867-3327b4b0591a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.340458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 31/66] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:04.345594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.346124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.346597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.373257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:04.374790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:04.465950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 36/67] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:04.545890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72477223-720e-4474-b867-3327b4b0591a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:04.603248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:04.676832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:04.709921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72477223-720e-4474-b867-3327b4b0591a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:04.725120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:04.738106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743175-a480-4f15-bdab-fe33130c4e6d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:04.766434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:04.767072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.767350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.767623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.781013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:04.781568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:04.782421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.782917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.783395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-6f8f8f24-ae5d-4673-bd42-421fce414d0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:04.788424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:04.789277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:04.790046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 32/68] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:04.790672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0694-af5c-4e96-baae-dcedf9622116 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.791515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0694-af5c-4e96-baae-dcedf9622116 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.791515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0694-af5c-4e96-baae-dcedf9622116 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.797408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 33/69] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:04.798519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f589267-47fa-484e-bc04-dd20fcfa7f30 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.798801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f589267-47fa-484e-bc04-dd20fcfa7f30 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.799082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f589267-47fa-484e-bc04-dd20fcfa7f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.805528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:04.812161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:04.813067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:04.813833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 37/70] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:04.814458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:04.822168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0f589267-47fa-484e-bc04-dd20fcfa7f30 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:04.823452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0f589267-47fa-484e-bc04-dd20fcfa7f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 34/71] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cf1c-7424-4c07-9865-b4b37a65a8cf None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cf1c-7424-4c07-9865-b4b37a65a8cf None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cf1c-7424-4c07-9865-b4b37a65a8cf None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 35/72] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:04.902745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26cf1c-7424-4c07-9865-b4b37a65a8cf None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:04.925786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668cf2-ea2f-475a-a7ea-a711486c35c2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:04.964030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:04.964824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:04.965109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:04.965385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:04.966054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:04.966365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:04.967736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:04.981162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 38/73] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:04.981162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:04.993877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.004404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.039136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26cf1c-7424-4c07-9865-b4b37a65a8cf None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.073869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.074486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.074766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.075039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.075711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:05.076019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.077450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980ce96-c814-4fb5-98db-0954ba311275 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.086663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 36/74] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.087880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.088167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.088477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.100803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6da74-814b-46b2-b4ab-eb345abd8a3d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.107671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:05.108955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:05.142558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.144113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.144560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.144973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.158001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:05.163929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:05.164835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 39/75] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:05.170920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.188616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 37/76] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:05.189469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.232898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.259705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.270073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccd7e6-0910-4216-adb3-adc4bb68f5c3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.300720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb835-73b7-4917-bb61-40f89da8be30 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.305980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.306621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.306906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.307187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.320831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.322314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.323944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 40/77] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.324764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.342140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.343000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.343427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.343836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.345378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.347317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.349101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 38/78] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.350571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.435377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa9a53-1035-4491-9542-7fbd058c9bd6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.463809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.468273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f6f92-81bb-41a1-98c8-6ae261057555 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.473468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.474084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.474364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.474637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.475298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:05.475611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.477018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.490995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 41/79] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.491801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.500920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.507184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.507824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.508106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.508432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.518938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:05.524167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:05.525056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:05.525834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 39/80] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:05.526456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.594095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc6629-04f6-4b39-8de2-6a4e7afff498 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.626014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf65d5-ec0d-4f21-9c8d-09c54ae464b7 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.627813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.628447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.628726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.628996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.659133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.659877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.660161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.660477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.682784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.694812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.696306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.718168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.728359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.729821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.791810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 42/81] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:05.792677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.833668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 40/82] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:05.834606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.894314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.906877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd647e-0ff4-42a7-931f-3d0de8428051 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.931150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:05.941387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4ae84-a693-440a-bec8-9d50f66f3d38 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:05.944375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.945060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.945313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.945611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.946767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.948370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.949989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 43/83] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.950755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:05.976756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:05.977384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:05.977662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:05.977933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:05.979059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:05.980525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:05.982077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 41/84] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:05.982822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.050636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaac64-3788-4654-87b2-35c1b35ec18c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.085733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.086411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.086697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.086781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f4c96-5aed-4d55-b816-ee52f617ec71 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.087177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.087620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:06.087925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.089339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.098108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.098913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.118355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.125310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.125951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.126231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.126508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.128518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.129983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.131614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 42/86] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:05 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.132380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.159973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.201537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6b5d-9313-417f-a1f7-4ddde9278dd8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.232882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c6cbd-a9f3-4f1e-b8d2-27f1e13f8cc0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.234425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.235059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.235336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.235610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.237343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.238787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.240422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 45/87] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.241182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.272128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.272755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.273031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.273300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.274006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.275435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.296652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 43/88] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:06 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:06.297332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.328481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.348574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502032-6c72-4521-84a3-02fe3680c98f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.388751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.389754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.390191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.390608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.399128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.418878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e36aa-0cb6-4646-a25c-fdcdd31c8d93 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.458288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.459211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.459506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.459986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.461204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.462670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.464272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 44/89] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.469240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.475159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.476607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.572209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.595615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 46/90] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:06.597289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.602780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834b9e5-b846-4d0a-bdd5-c3259c7e1ee9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.649820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.650588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.650878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.651157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.652075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:06.653117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.654417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.662675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 45/91] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.663827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.689090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.729038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9dbb2-3a14-4de2-a75b-8021c6e1d04f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.811389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439287c-3b9a-471e-9898-3ea82310e8f6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:06.813728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.814369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.814651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.815117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.816113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=6c403c92b6314d26999d2aed9104bf3f) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:06.816459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.817976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.831552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/role_assignments?user.id=6c403c92b6314d26999d2aed9104bf3f&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:06.832164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:06.854145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:06.854779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:06.855059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:06.855333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:06.953483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:06.954963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:06.977401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:06.993399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:07.064719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4f47d-fd0e-42c0-bed5-83e3702f93e8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:07.106212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 46/93] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:07.125592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:07.148974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:07.149631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:07.149909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:07.150185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:07.156940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:07.158416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:07.160869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 48/94] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:06 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:07.173269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:07.279812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c716e-9020-450b-b8a9-06df573c2e3e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:07.312509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:07.349845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:07.350779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:07.351267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:07.351709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] PATH_INFO: `/v3/projects/876c2813dec04e3f82f93b4404132beb` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:07.354243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] RBAC: Authorizing `identity:get_project(project_id=876c2813dec04e3f82f93b4404132beb)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:07.356897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:07.358844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 47/95] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:07 2022] GET /identity/v3/projects/876c2813dec04e3f82f93b4404132beb => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:07.365595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:07.390962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1b2de-7a5a-4d07-8d9a-9757d3a17385 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:07.425100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:07.461785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:07.462721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:07.463167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:07.463601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:07.464516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:07.465009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:07.467702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd60c36-edfe-4201-8955-74b7c0ffc3ce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:07.478434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 49/96] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:07 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:07.537911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eebde-7b26-4497-b047-26bc3f6818c0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:07.587876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:07.588822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:07.589270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:07.589707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:07.590589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=876c2813dec04e3f82f93b4404132beb, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:07.591065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:07.592780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad630-a58c-472a-99a6-869bfa2cfe37 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:07.604779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 48/97] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:07 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=876c2813dec04e3f82f93b4404132beb&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:09.170508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565349-f8be-4783-925e-72678a6943f1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:09.171044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565349-f8be-4783-925e-72678a6943f1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:09.171044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565349-f8be-4783-925e-72678a6943f1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:09.172539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 50/98] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:09.188858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60940e-3398-48d3-acd7-bc6cd07b1783 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:09.190284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60940e-3398-48d3-acd7-bc6cd07b1783 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:09.190986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60940e-3398-48d3-acd7-bc6cd07b1783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:09.215638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-eb60940e-3398-48d3-acd7-bc6cd07b1783 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:09.217284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-eb60940e-3398-48d3-acd7-bc6cd07b1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:09.292287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:09.326533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 49/99] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:09.339746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ae29d-1759-4deb-93e9-b65efd9c3353 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:09.340384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ae29d-1759-4deb-93e9-b65efd9c3353 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:09.340850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ae29d-1759-4deb-93e9-b65efd9c3353 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:09.342365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 51/100] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:09.347903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:09.348200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:09.348504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:09.372224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:09.373365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:09.454142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 50/101] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:09.771801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ae29d-1759-4deb-93e9-b65efd9c3353 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:09.933125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:09.986087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ae29d-1759-4deb-93e9-b65efd9c3353 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:10.083425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:10.083425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.083425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.083425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.129030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cc38e8ad-cfa5-4004-8168-59351d2f6de6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:10.134826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:10.135669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:10.136432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:10.140567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 52/102] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:10.145709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caee8cb-bc0c-4a43-a978-146aa3df2f78 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.146226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caee8cb-bc0c-4a43-a978-146aa3df2f78 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.146692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caee8cb-bc0c-4a43-a978-146aa3df2f78 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.148192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 53/103] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:10.157315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caee8cb-bc0c-4a43-a978-146aa3df2f78 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:10.217397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dcdc7-0e40-4aeb-ae92-cc5e04b306d3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:10.233662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:10.262193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:10.263326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.263817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.264303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.265791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:10.267525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa11f50-db3b-4d38-b6ee-0e80b4fba833 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:10.271448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 51/104] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:10.277704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.278976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.279470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.298169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:10.304443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:10.306096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:10.395334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 52/105] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:10.397000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:10.441966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caee8cb-bc0c-4a43-a978-146aa3df2f78 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:10.512032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:10.512678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.512959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.513235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.513900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:10.514201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:10.515588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840aa2-21fb-4f04-b4a5-51efdd27237c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:10.540964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 54/106] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:10.542264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006afe5-e965-44b1-986e-e669736150a3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.542551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006afe5-e965-44b1-986e-e669736150a3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.542823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006afe5-e965-44b1-986e-e669736150a3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.544205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 55/107] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:10.544768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006afe5-e965-44b1-986e-e669736150a3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:10.603845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82179bc-a1e8-4de6-90e3-5f9ac6ba2db3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:10.619508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:10.656798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:10.657750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.658251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.659424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.660569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:10.662571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f813a589-4aef-49db-aeee-a9b29c0517de None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:10.696880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 53/108] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:15:10 2022] POST /identity/v3/projects => generated 302 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:10.705387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.707205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.707696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.744235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006afe5-e965-44b1-986e-e669736150a3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:10.747401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:10.750542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:10.750709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:10.836978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:10.837631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:10.838545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:10.838831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] PATH_INFO: `/v3/users/11b9802a99af4a938bc601e3261753d9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:10.866861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 54/109] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:10.867574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:10.871513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] RBAC: Authorizing `identity:get_user(user_id=11b9802a99af4a938bc601e3261753d9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:10.873107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:10.875369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 56/110] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:10 2022] GET /identity/v3/users/11b9802a99af4a938bc601e3261753d9 => generated 289 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:10.881463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.015687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.031049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc4816-c93e-4ad4-addd-300942348d71 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.070588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.071226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.071503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.071768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.085489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-8c4560ae-f0bf-4893-a27f-56e3b1501d5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:11.091100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:11.091967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 55/111] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:11.097345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0738f-790d-4740-a7c2-f2b5ce8bc89b None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.098276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0738f-790d-4740-a7c2-f2b5ce8bc89b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.100583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0738f-790d-4740-a7c2-f2b5ce8bc89b None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.103017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 56/112] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:11.103996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0738f-790d-4740-a7c2-f2b5ce8bc89b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.121341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442fd2d-39c5-40cf-b853-c76e2c1b05b2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.163758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.164676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.165438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.165714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.203147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.231026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0738f-790d-4740-a7c2-f2b5ce8bc89b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.245876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.247470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.266997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.267527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.268450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.268896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.269758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:11.270062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.271444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736823b-e981-4f60-94cf-82f778386bc5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.279838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 57/113] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:11.281137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e34c4-7e50-48c3-8f77-bf1353805945 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.281421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e34c4-7e50-48c3-8f77-bf1353805945 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.281693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e34c4-7e50-48c3-8f77-bf1353805945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.300146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-854e34c4-7e50-48c3-8f77-bf1353805945 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:11.301344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-854e34c4-7e50-48c3-8f77-bf1353805945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:11.304979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 58/114] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114488-305f-41dc-b018-432fe34fa89e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114488-305f-41dc-b018-432fe34fa89e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114488-305f-41dc-b018-432fe34fa89e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 59/115] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:11.378593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114488-305f-41dc-b018-432fe34fa89e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.386836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 57/116] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:11.387755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.461553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.495672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114488-305f-41dc-b018-432fe34fa89e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.505854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd705a66-db73-4db1-82b6-d6d0120b6769 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.530430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.530986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.531356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.531592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.539963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.543051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:11.546257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.547067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.547481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.548124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-3780e746-a681-4430-ab03-5e247c83a9f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:11.548821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:11.550231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:11.551189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:11.552997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.554682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.555935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 60/117] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:11.556823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 58/118] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:11.557255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.557537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.557807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.558347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.580425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:11.581665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:11.667469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc9dbf-76c5-4c6d-8ece-0cb34fdd3435 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.671144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 61/119] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:11.671996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.700080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.707131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.708144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.708730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.709231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.710522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.712197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.714577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 59/120] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:11.715522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.789727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ea4f2-7498-462f-a1f5-b29320f76caf None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.823450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.829903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.831176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.831523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.831880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.832752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:11.833231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.834683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.842168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d37141-e02b-4a01-9b2d-050d7e8aaa0d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.849579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 62/121] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:11.849579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.886647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.887673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.888236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.888743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:11.890171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:11.891858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:11.895115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 60/122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:11.896104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:11.954852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0eb34-1073-4676-a132-2c4d06594330 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:11.971691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:11.992689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:11.993327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:11.993603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:11.993875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.005202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:12.008108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904d2cd-ceb8-4f4f-8750-2821c8b65414 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:12.010047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:12.010873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:12.011723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:12.016079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 63/123] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:12.016875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.046986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.047526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.047809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.048079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.048859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=11b9802a99af4a938bc601e3261753d9) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:12.049191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.050615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.056293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 61/124] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:11 2022] GET /identity/v3/role_assignments?user.id=11b9802a99af4a938bc601e3261753d9&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.056827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.086597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.165963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d0b2d-4cb8-4abc-87fa-738bfe3e34d2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.210825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a108-9b20-4fb7-9f26-9b4d7cafe72d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.218344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.219337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.219835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.221048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.222674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.225490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.227921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 64/125] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.229652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.254432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.260937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.261702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.262055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.262400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/7d17d179d17e40648c50edcb7eb2560d/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.264584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, user_id=7d17d179d17e40648c50edcb7eb2560d, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.266145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.283096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 62/126] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:12 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/7d17d179d17e40648c50edcb7eb2560d/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:12.284504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.433598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56900ed4-ba1a-4e13-95f3-bcad0d886c85 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.442654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940708a4-69a8-4887-a266-0ea82a14cb3f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.480827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.482307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.483136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.483589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.484019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.484917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:12.485395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.488234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.500897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 63/127] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.500897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.517074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.517818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.518083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.518601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.529992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:12.536051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:12.536944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:12.537721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 65/128] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:12.538353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.573794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.638884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3d95-51af-417e-8467-734191072b17 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.680856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.681504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.682042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.682327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.683491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.685078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.687294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 64/129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.688056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.723883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1b118-5e45-4bd7-9f03-c47cb8e6baf0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.790822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.791542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.791823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.792095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.793948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.795834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.798510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 66/130] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.799135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:12.872389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff087-bae4-472e-b04b-77c4903612d1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:12.903669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.913209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:12.942780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:12.943410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:12.943691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:12.943962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:12.944668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:12.944972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:12.946349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:12.965617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 65/131] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:12.966289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:13.000924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da87464-4d21-4e1c-8766-ec88308f42aa None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.053532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.054213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.054550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.054828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/6c403c92b6314d26999d2aed9104bf3f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.059299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, user_id=6c403c92b6314d26999d2aed9104bf3f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.060785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.089418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 67/132] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:12 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/6c403c92b6314d26999d2aed9104bf3f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:13.090063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:13.147613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e665-c3c4-49aa-9132-5ff6f329ee21 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.209425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.209995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.210277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.210612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.214290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.215929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.217900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 66/133] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:12 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:13.224135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:13.244238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba160fd-57b0-4509-81af-a4838aca3859 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.256688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:13.297752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.298528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.299785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.300374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.302842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.306019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.309548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 68/134] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:13 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:13.316264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:13.341904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:13.450491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de31b3-08ec-445d-aeb5-796574cff427 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.468447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623781d8-7032-4d98-a214-6e77f7593bc0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.497332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.498015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.498302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.498584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/7d17d179d17e40648c50edcb7eb2560d/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.500850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, user_id=7d17d179d17e40648c50edcb7eb2560d, role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.503363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.515324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.516093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.516634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.517518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] PATH_INFO: `/v3/projects/876c2813dec04e3f82f93b4404132beb` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.519163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] RBAC: Authorizing `identity:get_project(project_id=876c2813dec04e3f82f93b4404132beb)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.522045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e4e5e-714f-485a-bd73-6838f4023590 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.524597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 69/135] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:13 2022] GET /identity/v3/projects/876c2813dec04e3f82f93b4404132beb => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:13.526573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:13 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/7d17d179d17e40648c50edcb7eb2560d/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:15:13.527155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b935c82-2f59-44f9-bd22-d9dd962352b2 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.527757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b935c82-2f59-44f9-bd22-d9dd962352b2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.528790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b935c82-2f59-44f9-bd22-d9dd962352b2 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.530501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 70/137] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:13.530674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:13.551009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.552385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.552911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.586118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:13.591469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:13.728023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:13.784205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 71/138] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:13.798008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ee25-202f-4490-89ee-02ad50024113 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:13.809705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:13.838679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:13.839344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:13.839621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:13.839894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] PATH_INFO: `/v3/projects/876c2813dec04e3f82f93b4404132beb/users/7d17d179d17e40648c50edcb7eb2560d/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:13.841891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] RBAC: Authorizing `identity:create_grant(project_id=876c2813dec04e3f82f93b4404132beb, user_id=7d17d179d17e40648c50edcb7eb2560d, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:13.843310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:13.857964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 68/139] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:13 2022] PUT /identity/v3/projects/876c2813dec04e3f82f93b4404132beb/users/7d17d179d17e40648c50edcb7eb2560d/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:14.246918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:14.362599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f15d-5a92-4724-a8ae-019a0673da5b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:14.405616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:14.406263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:14.406549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:14.406878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:14.412972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:14.414442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d0210-ccbc-4bac-ba48-9db399150e11 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:14.416052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 72/140] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:14.421751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:14.542220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702f183-34fd-4cab-825b-3a712ac4c80a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:14.576868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:14.577493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:14.577778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:14.578058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:14.578806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:14.580434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e68531-b548-4939-92fe-582270cbf71e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:14.611273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 69/141] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:14 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:14.892530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb5ae-69fb-48a4-a639-7771908c12d8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:14.893221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb5ae-69fb-48a4-a639-7771908c12d8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:14.893604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb5ae-69fb-48a4-a639-7771908c12d8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:14.894874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 73/142] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:14.900311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2d6a5-8949-4023-86a6-a48fc3cfb470 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:14.900662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2d6a5-8949-4023-86a6-a48fc3cfb470 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:14.900949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2d6a5-8949-4023-86a6-a48fc3cfb470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:14.925442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-37b2d6a5-8949-4023-86a6-a48fc3cfb470 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:14.926783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-37b2d6a5-8949-4023-86a6-a48fc3cfb470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:14.951417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:15.014586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 70/143] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:15.395008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757d07-7286-4822-be72-b11a482957c8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:15.395297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757d07-7286-4822-be72-b11a482957c8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:15.395598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57757d07-7286-4822-be72-b11a482957c8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:15.397014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 74/144] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:15.402489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:15.403022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:15.403509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:15.429313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:15.430894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:15.514557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57757d07-7286-4822-be72-b11a482957c8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:15.524055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 71/145] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:15.586119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:15.654874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57757d07-7286-4822-be72-b11a482957c8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:15.702419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:15.703415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:15.703904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:15.704405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:15.722772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:15.728150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:15.729068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:15.729888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:15.732137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 75/146] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:15.738136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:15.855652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:15.875080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477fae9-b5a9-4dc1-a2ee-ff5a43e647ef None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:15.914043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:15.914630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:15.914907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:15.915174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:15.915847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:15.916146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:15.917563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:15.929737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 72/147] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:15.935297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.071150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f166-c05a-43a2-bdaf-7fb65384e874 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.073106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:16.118708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:16.119352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.119881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.120171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] PATH_INFO: `/v3/users/11b9802a99af4a938bc601e3261753d9` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.124967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=11b9802a99af4a938bc601e3261753d9)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:16.126415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:16.128725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 76/148] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:15 2022] GET /identity/v3/users/11b9802a99af4a938bc601e3261753d9 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:16.135029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.158027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.327107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845d2f5-18e9-40fc-aebb-6748ee277b8d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.354010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec425ae-8dcf-4c21-97c1-1953c5638ec2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.401328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:16.401981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.402263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.402538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.409987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:16.410648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.410923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.411188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.412368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:16.413828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:16.415783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 73/149] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:16 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:16.421319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.426889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:16.431830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:16.432743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:16.433516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 77/150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:16.440947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.473399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:16.512618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:16.666606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eadd3-84d7-4f15-bb32-82dbc3cd9a91 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.679516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386cd2c-107e-4346-9606-9e129259bfdc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.745796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:16.746423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.746705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.746978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.747652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:16.747953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:16.749358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe2e4-3422-4709-840f-5fe535356c3a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:16.750407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:16.751089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.751359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.751627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/11b9802a99af4a938bc601e3261753d9/roles/906b85ca0f0244f2ada233c8899f62f6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.755921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, user_id=11b9802a99af4a938bc601e3261753d9, role_id=906b85ca0f0244f2ada233c8899f62f6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:16.757399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cea9f-d6a3-43d1-9695-1eed282ea3b5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:16.762973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 78/151] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:16.763814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587de95-1ac4-4194-88c1-4a835aab4cf0 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.764099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587de95-1ac4-4194-88c1-4a835aab4cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.768679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587de95-1ac4-4194-88c1-4a835aab4cf0 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.776556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 79/152] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:16.777405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587de95-1ac4-4194-88c1-4a835aab4cf0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:16.778605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 74/153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:16 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/users/11b9802a99af4a938bc601e3261753d9/roles/906b85ca0f0244f2ada233c8899f62f6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:16.781985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5188cb2-9773-45f9-9239-073f4c249ce8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.782269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5188cb2-9773-45f9-9239-073f4c249ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.782548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5188cb2-9773-45f9-9239-073f4c249ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.821158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f5188cb2-9773-45f9-9239-073f4c249ce8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:16.822430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f5188cb2-9773-45f9-9239-073f4c249ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:16.939348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:16.940957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 75/154] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:16.943827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b62a-ba8d-40f8-9f90-945aed0b5dd2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.944060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b62a-ba8d-40f8-9f90-945aed0b5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.944385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b62a-ba8d-40f8-9f90-945aed0b5dd2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.946804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 76/155] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:16.952837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:16.953131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:16.953376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:16.961057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587de95-1ac4-4194-88c1-4a835aab4cf0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:16.973859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:16.975118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:16.980649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:17.000678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:17.001369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:17.001644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:17.001912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:17.003197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:17.004723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:17.006750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 80/156] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:16 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:17.011888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.126789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 77/157] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:17.169592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93bf7-15fe-44d6-b623-bc35ca787ce0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:17.211303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:17.211926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:17.212203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:17.212562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:17.245174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:17.296177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:17.297672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:17.402720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 81/158] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:17.408509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.513820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447efb50-6fab-49b6-b738-1004fcc374c9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:17.517016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.565980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:17.567102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:17.567500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:17.567776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:17.570115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:17.571885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:17.573914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 78/159] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:17.580852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:17.637882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:17.723190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597b46-8f9d-4f41-8ba5-40a889857879 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:17.726710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe631-ea0a-48c1-a02f-878bbf0d36c2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:17.786469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:17.786985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:17.787270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:17.787541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:17.788502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:17.788813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:17.790629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:17.797224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 79/160] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:17.798251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.818956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:17.820095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:17.820633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:17.821118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:17.853239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:17.858758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:17.859690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:17.865003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:17.867239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 82/161] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:17.874574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:17.931902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:17.991702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c69160-7106-4494-9950-5429080e5e9c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.062968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.064179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.064578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.064844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.099417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:18.105610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:18.106567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:18.107364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 80/162] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:18.115182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:18.176696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6bc8b-1d6d-46ed-8923-ee4ccdfbcfa9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.207312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:18.225141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.225760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.226037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.226310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.226987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:18.227288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:18.228728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:18.240931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 83/163] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:18.246920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:18.347788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d675dbc-c5f2-41e2-8c06-12a127cb6796 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.422067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:18.437692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.438572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.438859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.439131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.440108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:18.440440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:18.441856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad6a1a-c82f-40d6-94cf-36eafc65e6de None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:18.466602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 81/164] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:18.470164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd9f7-6638-41ea-a67b-381be13438fb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.470455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd9f7-6638-41ea-a67b-381be13438fb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.471411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd9f7-6638-41ea-a67b-381be13438fb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.475135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 82/165] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:18.475887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafd9f7-6638-41ea-a67b-381be13438fb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:18.487015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfbebd-1ed5-4ea1-baba-477c95df8165 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.531978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.534785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.535067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.535344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] PATH_INFO: `/v3/users/6c403c92b6314d26999d2aed9104bf3f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.536621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] RBAC: Authorizing `identity:get_user(user_id=6c403c92b6314d26999d2aed9104bf3f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:18.538083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bb0c-d077-45fb-94f6-e0ed12f94feb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:18.539956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 84/166] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:18 2022] GET /identity/v3/users/6c403c92b6314d26999d2aed9104bf3f => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:18.541013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.541302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.541597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.570488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:18.571751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:18.625814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:18.647417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 85/167] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:18.648302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:18.723993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafd9f7-6638-41ea-a67b-381be13438fb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.791034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.791803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.791979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.792263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.793994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:18.796432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:18.800277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 83/168] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:18 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:18.810772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:18.846263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721603f5-92f3-42c9-b672-e9b6d7fc7411 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:18.893251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:18.893757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:18.894078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:18.894417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:18.940473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.008841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.010678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.012192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.031887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3170aef-b94a-4629-b751-293c91b8f872 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.101444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.102057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.102334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.102603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.168892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 86/169] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:19.175330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.215718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.217355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.259560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.285405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7a4fe-0296-4ebe-8c4a-82a4cea08b28 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.323131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.323765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.324068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.324345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.325495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.326935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.328661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 87/170] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:19.329423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.331137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.415063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 84/171] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:19.416018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.438699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea03c7-0b1b-4e34-88da-e70f1ca7e85a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.480197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.480854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.481130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.484280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.505103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:19.510049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:19.511010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:19.511809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 88/172] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:19.513406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.526579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.620092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a12231-d0d1-467b-9515-9ad188e2a063 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.670936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.671640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.671920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.672187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.672946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=6c403c92b6314d26999d2aed9104bf3f) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:19.673251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.675349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.688691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 85/173] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/role_assignments?user.id=6c403c92b6314d26999d2aed9104bf3f&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:19.691121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.715699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a60b7-b9eb-4487-9c7b-68a27f9242d5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.737293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:19.799962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.800674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.800952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.801221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.802654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.804106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e874338-ea40-4961-ad32-a1abc6b1b220 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.805803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 89/174] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:19.807005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62064-d5cf-489c-8e8c-3cc79fdd21c5 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.807282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62064-d5cf-489c-8e8c-3cc79fdd21c5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.807550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62064-d5cf-489c-8e8c-3cc79fdd21c5 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.810194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 90/175] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:19.817341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42d80-6365-4fd2-92e9-aee0816b9908 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.817675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42d80-6365-4fd2-92e9-aee0816b9908 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.817885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42d80-6365-4fd2-92e9-aee0816b9908 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.819297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 91/176] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:19.820499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88038d-4dbb-4522-8089-b97407e4f446 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.820770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88038d-4dbb-4522-8089-b97407e4f446 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.821033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88038d-4dbb-4522-8089-b97407e4f446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.865030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ce88038d-4dbb-4522-8089-b97407e4f446 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:19.866249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ce88038d-4dbb-4522-8089-b97407e4f446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:19.889883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1d958-3e75-4790-aaeb-f5e59b023163 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:19.959417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:19.960040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.960338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.961149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:19.961821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:19.962123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:19.963570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:19.975795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 86/177] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:19.981113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:19.985484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 92/178] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:19.987771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:19.988062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:19.988374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.005604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.020407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:20.021604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:20.084756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.105579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc748dca-8296-4e75-a09e-837cfbd18f40 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.146506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.147116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.147442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.147714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.148462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:20.148767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:20.150801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:20.158893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 87/179] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:19 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:20.159780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.175686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 93/180] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:20.290428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a332b-cf59-4d90-ba60-ba2805f42de1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.339977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.340660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.340940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.341217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] PATH_INFO: `/v3/users/7d17d179d17e40648c50edcb7eb2560d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.342388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d17d179d17e40648c50edcb7eb2560d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:20.343866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:20.345493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 88/181] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/users/7d17d179d17e40648c50edcb7eb2560d => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:20.351583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.433334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.466491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.503757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176bfa9-a822-4dbc-850e-f9aad33f42d8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.580848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.581777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.582204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.582619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.588648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.621128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7359c6-9a6a-45dc-9a3a-d192c0eed5f2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.660098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.660768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.661047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.661315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.673414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:20.675758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c785d-84e8-424e-ad9b-8e84e6e804da None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:20.677540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:20.683641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:20.685027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 89/182] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:20.685825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.776045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 94/183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4269-ebc7-4b4e-9c2d-723bcc026133 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4269-ebc7-4b4e-9c2d-723bcc026133 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4269-ebc7-4b4e-9c2d-723bcc026133 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 95/184] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:20.800369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4269-ebc7-4b4e-9c2d-723bcc026133 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.807130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac2b0c-32d0-4088-b507-15ba63abbc16 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.843488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:20.849309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.849964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.850238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.850510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.864332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f4b03cf6-83db-4af2-be2e-3c1f9710ba62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:20.871890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:20.872912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:20.873813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:20.876388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 90/185] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:20.877948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.878413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.878692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.897158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:20.898359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:20.921773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4269-ebc7-4b4e-9c2d-723bcc026133 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:20.958340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:20.958948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:20.959229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:20.959503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:20.960169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:20.960513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:20.961914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:20.968227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 91/186] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:20.969896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 96/187] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:20.971443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:20.973207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.055105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.072979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.077354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56853a86-b7ed-4014-a95c-99272a06bf8e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.091663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724aac6-c6a1-473c-baa2-ff9890599b0a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.120583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.121469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.122159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.122444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.123280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:21.123587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.125601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424090a-48ef-46e7-8a03-e7f0f1eb17c1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.130262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.131324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.131609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.132107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] PATH_INFO: `/v3/projects/876c2813dec04e3f82f93b4404132beb` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.133197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] RBAC: Authorizing `identity:get_project(project_id=876c2813dec04e3f82f93b4404132beb)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.134010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 97/188] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.134983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9699d5-5d50-41d5-b5f4-42bac25afedb None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.135267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9699d5-5d50-41d5-b5f4-42bac25afedb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.135540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9699d5-5d50-41d5-b5f4-42bac25afedb None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.136773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 98/189] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:21.140503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9699d5-5d50-41d5-b5f4-42bac25afedb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.140699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.143607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 92/190] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:20 2022] GET /identity/v3/projects/876c2813dec04e3f82f93b4404132beb => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.144398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.246019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9699d5-5d50-41d5-b5f4-42bac25afedb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.253439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad55303-7c94-4356-a194-10bf4ac5c2fa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.282720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.283859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.284131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.284447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.289080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.289706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.289980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.290251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.292127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.299840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.301316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22fcbf-4dca-4c8b-a5bc-a334425c1ee3 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.302945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 99/191] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.304307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.304550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.304550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.311027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.311027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.311027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 93/192] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.311027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.326087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:21.327258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:21.341154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.395298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 100/193] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:21.396053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.409493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e49d-42e4-4879-a697-6915a313c937 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.446543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.447135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.447406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.447670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.448410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=876c2813dec04e3f82f93b4404132beb, user.id=7d17d179d17e40648c50edcb7eb2560d) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:21.448711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.450137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.456976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 94/194] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/role_assignments?user.id=7d17d179d17e40648c50edcb7eb2560d&scope.project.id=876c2813dec04e3f82f93b4404132beb&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.457743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.525788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.531586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dd5b2-92ac-4af0-80ec-e9b72c4ae0a6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.577400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd366394-b78d-465b-9eca-cb1a97b67383 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.583771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.584447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.584723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.584992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.586228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.587740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.589619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 101/195] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:21.595269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.616835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.630645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.631670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.632098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.632840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.716747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.718827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b915f7-9ea9-4ade-b7bf-fb4ddb5535a9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.718981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.754502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.755105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.755375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.755639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.775686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.820379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 95/196] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:21.821188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.822936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.824421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.840880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:21.917576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c269b-a78e-4c31-8d3e-ffdc9fad6d8b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:21.919479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 102/197] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:21.920334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.951663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:21.952282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:21.952561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:21.952830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:21.953477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:21.954904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:21.976492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 96/198] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:21 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:21.977198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:21.994616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.028804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9718d43-3a05-45e2-bf82-d3acc39ab1b6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.048477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.071951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.072634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.072907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.073177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.090100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:22.092632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427164f-a5d4-4f42-96fb-6120283e14ce None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:22.097934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:22.098895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:22.099700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 103/199] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:22.100417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.141284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.141999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.142279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.142671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.144095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.145587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.147234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 97/200] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:21 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.147994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.231678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7b3a-ee20-4949-a5de-b15eca2cacee None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.248870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.271400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aadeb-8fec-4c14-a1ef-20d1e49ecf1b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.276648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.277569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.278044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.278533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.280899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.282626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.285706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 104/201] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.286620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.314893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.315619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.315904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.316181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.316890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:22.317197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.319183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.332373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 98/202] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.334571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.347266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.421807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e83c-dd5b-4c62-9f17-c8ee174b1afe None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.451258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657244-c3ef-4250-9127-68e018b549f0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.469773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.471675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.472204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.472706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.473613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4fb8c9b617d44dc797a1d24915c1c505) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:22.474639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.476909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.485634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 105/203] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/role_assignments?user.id=4fb8c9b617d44dc797a1d24915c1c505&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.486242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.493294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.494990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.494990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.494990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.495505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4fb8c9b617d44dc797a1d24915c1c505) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:22.495814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.497315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.504691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 99/204] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/role_assignments?user.id=4fb8c9b617d44dc797a1d24915c1c505&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.621162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b6e1-3c27-4c18-ab9b-07ebcf6f5ae4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.633020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.672849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.673495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.674153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.674434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] PATH_INFO: `/v3/users/11b9802a99af4a938bc601e3261753d9` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.675749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] RBAC: Authorizing `identity:get_user(user_id=11b9802a99af4a938bc601e3261753d9)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.677255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.679138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 106/205] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/users/11b9802a99af4a938bc601e3261753d9 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:22.684839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:22.697742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:22.812063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d90b9-408e-4852-af36-a39147453ffc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:22.864938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:22.865551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:22.865830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:22.866097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:22.958277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:22.959734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:22.971343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:23.103678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 100/206] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:23.109395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:23.262124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d640fd-f6a3-49d3-99d3-5736652ef2dc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:23.314197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:23.314916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:23.315177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:23.315448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:23.316885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:23.318313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2da51-f20b-41f2-a5f4-5dacfdb06333 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:23.319917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 107/207] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:23 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:23.327679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:23.481921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52eb62f-8ead-406d-b3be-1b99ffcb93a8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:23.490381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:23.537958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:23.538963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:23.539451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:23.539917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:23.540875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, scope.project.id=f415ede09edd4a15803a41d7f0ac862c, user.id=11b9802a99af4a938bc601e3261753d9) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:23.541379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:23.543006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c69a7-a72c-46fd-a241-9ca2f0ae9319 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:23.554383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 101/208] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:23 2022] GET /identity/v3/role_assignments?user.id=11b9802a99af4a938bc601e3261753d9&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:24.504994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e72bd-ed9c-4423-a621-f5e54cf80eea None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.506006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e72bd-ed9c-4423-a621-f5e54cf80eea None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.506496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e72bd-ed9c-4423-a621-f5e54cf80eea None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:24.508059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 108/209] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:24.513644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6ee6-4775-486a-8d6a-67a34b8ec2df None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.514449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6ee6-4775-486a-8d6a-67a34b8ec2df None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.514953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6ee6-4775-486a-8d6a-67a34b8ec2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:24.534627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d39a6ee6-4775-486a-8d6a-67a34b8ec2df None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:24.536134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d39a6ee6-4775-486a-8d6a-67a34b8ec2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:24.604546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 102/210] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:24.853911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbc22-3ce3-4817-9fd6-d5636b944f77 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.854585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbc22-3ce3-4817-9fd6-d5636b944f77 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.855082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbc22-3ce3-4817-9fd6-d5636b944f77 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:24.857090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 109/211] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:24.866384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c0e4-6862-485f-a0a9-11925d99e02c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.867022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c0e4-6862-485f-a0a9-11925d99e02c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.867513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c0e4-6862-485f-a0a9-11925d99e02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:24.883795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cbc22-3ce3-4817-9fd6-d5636b944f77 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:24.889196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-df75c0e4-6862-485f-a0a9-11925d99e02c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:24.890359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-df75c0e4-6862-485f-a0a9-11925d99e02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:24.934541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:24.969652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 103/212] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:24.977197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78831b1e-f29a-485f-9185-3d945035a7f0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.977463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78831b1e-f29a-485f-9185-3d945035a7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.977735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78831b1e-f29a-485f-9185-3d945035a7f0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:24.979223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 104/213] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:24.986932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da868-26b0-4404-ae59-c614e02411fc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:24.987291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da868-26b0-4404-ae59-c614e02411fc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:24.987472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da868-26b0-4404-ae59-c614e02411fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.007087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cbc22-3ce3-4817-9fd6-d5636b944f77 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.013535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6c5da868-26b0-4404-ae59-c614e02411fc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:25.014755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6c5da868-26b0-4404-ae59-c614e02411fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:25.051289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.054452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943878da-287c-4470-8479-95061bbd5553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.055546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943878da-287c-4470-8479-95061bbd5553 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.056124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943878da-287c-4470-8479-95061bbd5553 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.056629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943878da-287c-4470-8479-95061bbd5553 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.072080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943878da-287c-4470-8479-95061bbd5553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-943878da-287c-4470-8479-95061bbd5553 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:25.078502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:25.079433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:25.081036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 110/214] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:25.089938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943878da-287c-4470-8479-95061bbd5553 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.109007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 105/215] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:25.117964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de08a4-771b-46c9-861e-d89d4235e20c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.118466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de08a4-771b-46c9-861e-d89d4235e20c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.118930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de08a4-771b-46c9-861e-d89d4235e20c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.124359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 106/216] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:25.125629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.126073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.126514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.147349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:25.148810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:25.203526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943878da-287c-4470-8479-95061bbd5553 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.207464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.238497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 107/217] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:25.245103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.245714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.246040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.246317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.247029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:25.247334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.248789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.258225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 111/218] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:25.263680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.263812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.367875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.373093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111943a-81dd-4e55-b8ec-7317fe6cfd58 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.391346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3fe36-b834-488f-aedb-897cf9d57da1 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.410559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.411199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.411477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.411750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] PATH_INFO: `/v3/users/bc94a478ebd64d63b14964e895c29e03` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.425685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] RBAC: Authorizing `identity:get_user(user_id=bc94a478ebd64d63b14964e895c29e03)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.427236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.428342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.429086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 108/219] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/users/bc94a478ebd64d63b14964e895c29e03 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:25.429191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.429280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.429503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.430613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.432051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.436291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 112/220] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:25.436441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.436546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.473101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.538787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30440786-f0d6-427f-b897-4998d4230372 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.545945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48536-af8c-469b-a73b-ea2e65486e8b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.571873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.572532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.572813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.573083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.580971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.581577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.581855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.582126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.583925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:25.589130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:25.590022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:25.590826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 109/221] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:25.591867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.603458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.650777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.652172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.685548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.694226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bad4-b0bf-46ff-802b-3e29c2a0f6a7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.728041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.728671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.728944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.729211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.729852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.731291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.753012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 110/222] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:25 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:25.753833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.759221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 113/223] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:25.759913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.836707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.881405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e835d-e18d-4a7c-bdcf-661d3c6d26e5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.881615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42375579-b2a3-4fb8-bd99-8b9f2f029b92 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:25.920530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.920744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:25.921070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.921171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:25.921351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.921445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:25.921614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.921706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:25.922283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:25.922587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:25.924811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:25.928296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:25.935089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:25.942007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 114/224] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:25.942007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:25.942235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:25.943448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:25.944243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 111/225] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:25.946822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.070724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2d491-dc57-41f7-9ada-dcc7b113f0d7 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.085658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2503ab-8284-4cd5-861b-ffdc893949ba None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.123127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:26.123606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.124390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.124581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.124867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.127584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.129028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.130855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 115/226] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.131446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.139083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.139730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.140009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.140324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.141482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.143004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.145611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 112/227] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:25 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:26.146383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.204759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:26.246111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8ed5-061d-4c78-b78a-1dbb3cd2c6d0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.283656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a177a-bcc5-4800-a041-e6bacc55ea8d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.289476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.290407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.291383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.291865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.292776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:26.293759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.296015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.312851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 116/228] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.313289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.329834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.330907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.331186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.331459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.332244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=bc94a478ebd64d63b14964e895c29e03) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:26.332597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.334111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.344278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 113/229] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:26 2022] GET /identity/v3/role_assignments?user.id=bc94a478ebd64d63b14964e895c29e03&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.344278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.460979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:26.506905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb1df-a5df-45b5-b789-fe54580bbd68 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.544439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:26.557917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142962a0-accf-4006-b5e4-a41f3d040503 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.560552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.561204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.561481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.561751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.562998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.564999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.566675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 117/230] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:26 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.573144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.625954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.627538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.628007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.628489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.629992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.633098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.635829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 114/231] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:26 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.644138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.763451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce97ef0-88a0-4551-8044-373e08f2f0af None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.780834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56d6e-ee0b-49d0-95e7-5e11fe034f8c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:26.789736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:26.802726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.803744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.804240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.804727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.806106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.807844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.809674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 115/232] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:26 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:26.821018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:26.852843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:26.854033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:26.855465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:26.856226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4fb8c9b617d44dc797a1d24915c1c505/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:26.861709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, user_id=4fb8c9b617d44dc797a1d24915c1c505, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:26.864390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a61f2-e7e5-469b-9724-9998061a8f30 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:26.890301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 118/233] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:26 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4fb8c9b617d44dc797a1d24915c1c505/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:15:26.971973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2ee4d-ec52-42b5-ab02-7f1c2292bf39 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:27.012636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:27.013829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:27.014441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:27.014929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4fb8c9b617d44dc797a1d24915c1c505/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:27.018166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, user_id=4fb8c9b617d44dc797a1d24915c1c505, role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:27.019909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07f15-33be-4f04-9f01-4899e23b09c0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:27.042086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 116/234] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:26 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4fb8c9b617d44dc797a1d24915c1c505/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:28.301009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b73fb-2cf5-4a05-9058-5cd7564c741c None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:28.301513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b73fb-2cf5-4a05-9058-5cd7564c741c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:28.301587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b73fb-2cf5-4a05-9058-5cd7564c741c None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:28.303131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 119/235] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:28.308441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae0a4-8ee4-48d5-968b-ea8ccbcca0ef None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:28.308847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae0a4-8ee4-48d5-968b-ea8ccbcca0ef None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:28.309010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cae0a4-8ee4-48d5-968b-ea8ccbcca0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:28.337218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-97cae0a4-8ee4-48d5-968b-ea8ccbcca0ef None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:28.338568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-97cae0a4-8ee4-48d5-968b-ea8ccbcca0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:28.399866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:28.430945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 117/236] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:28.546733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781fa26-68eb-4b06-889c-0553a04dbadd None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:28.547356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781fa26-68eb-4b06-889c-0553a04dbadd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:28.547831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781fa26-68eb-4b06-889c-0553a04dbadd None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:28.550447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 120/237] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:28.555859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:28.556419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:28.556917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:28.581217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:28.582709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:28.674237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 118/238] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:28.704852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781fa26-68eb-4b06-889c-0553a04dbadd None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:28.765423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:28.831344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781fa26-68eb-4b06-889c-0553a04dbadd None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:28.877074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:28.877719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:28.878048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:28.878381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:28.890313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:28.895693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:28.896619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:28.897402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 121/239] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:28.902917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.013823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f35a00-2d96-4958-92f9-9b8101b405ea None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.026036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.036613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.055625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.056238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.056535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.056803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.057460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:29.057757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:29.059130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c3f8-89fe-4a5a-8f19-296b2d21eb33 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:29.068195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 119/240] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:29.069423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bcf37-e46f-4c13-96ee-39a209461ea3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.069700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bcf37-e46f-4c13-96ee-39a209461ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.069969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bcf37-e46f-4c13-96ee-39a209461ea3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.071413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 120/241] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:29.075702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bcf37-e46f-4c13-96ee-39a209461ea3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.170118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352a70-f9ea-4505-92f3-ed1b6c8d5f35 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.184581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bcf37-e46f-4c13-96ee-39a209461ea3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.185280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.207829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.208851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.209328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.209844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.211335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:29.213131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c839659-d20b-45b3-ab61-4c63676fca23 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:29.215440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 122/242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:29.216966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.217485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.217953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.226505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.227135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.227410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.227677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] PATH_INFO: `/v3/users/4a868d5135a348c7963cbe3093d4f9ec` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.239418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:29.241115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:29.242359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] RBAC: Authorizing `identity:get_user(user_id=4a868d5135a348c7963cbe3093d4f9ec)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:29.243953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:29.246163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 121/243] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/users/4a868d5135a348c7963cbe3093d4f9ec => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:29.250764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.287266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.320566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 123/244] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:29.322146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.373836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad808-08a6-4d32-b79f-339215eb8926 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.415810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.418026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.418311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.418620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.419274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:29.420747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:29.458452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 122/245] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:29 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:29.506136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dfb90-72da-432a-9295-016e03bab160 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.533200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.565526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.566063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.566335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.566598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.650243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.671102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:29.672617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:29.790561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.807495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956b43b-edc4-4488-b49e-16bf45cefc00 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:29.828870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 124/246] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:29.834979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.851343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:29.852906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:29.853993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:29.854100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:29.854360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:29.872756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:29.879408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:29.880423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:29.882710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 123/247] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:29.889592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:29.974001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce779bd-e91a-4679-bd1e-2c5caf6629ed None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.021532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d153-0cca-4341-b3f7-054eecf3ff05 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.023532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.024171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.024536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.024801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.027189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.028683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ae9a5-863d-4930-a815-2561cabc0d00 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.030446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 125/248] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:30.031450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f537c-1210-4f0c-96b4-f4f7404e2c66 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.031726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f537c-1210-4f0c-96b4-f4f7404e2c66 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.031994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f537c-1210-4f0c-96b4-f4f7404e2c66 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.034215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 126/249] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:30.035218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5c71e-7f08-4748-ba3a-7a8bf3ed3591 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.035496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5c71e-7f08-4748-ba3a-7a8bf3ed3591 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.035760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5c71e-7f08-4748-ba3a-7a8bf3ed3591 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.041020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 127/250] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:30.042130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b1f8-ace7-4329-a23c-94abf8b8a267 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.042410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b1f8-ace7-4329-a23c-94abf8b8a267 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.042941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b1f8-ace7-4329-a23c-94abf8b8a267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.067705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b0e8b1f8-ace7-4329-a23c-94abf8b8a267 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:30.068908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b0e8b1f8-ace7-4329-a23c-94abf8b8a267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:30.071648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.072973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.073461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.073930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.075208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:30.076031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.077784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.090420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 124/251] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:30.091650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:30.105505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:30.167183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 128/252] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:30.169900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.170392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.170862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.194530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:30.196459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:30.200573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:30.281523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 129/253] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:30.283437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:30.310215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b6b6-16e0-4088-a97b-f7f2c1be14ad None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.406434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.406434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.407142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.407142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.407470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4a868d5135a348c7963cbe3093d4f9ec) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:30.407746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.409844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.415938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 125/254] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:30 2022] GET /identity/v3/role_assignments?user.id=4a868d5135a348c7963cbe3093d4f9ec&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:30.426436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fe6bf-d893-4cbb-a76a-1493ad29bd2c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.469090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.470190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.471290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.471823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] PATH_INFO: `/v3/users/bc94a478ebd64d63b14964e895c29e03` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.473322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] RBAC: Authorizing `identity:get_user(user_id=bc94a478ebd64d63b14964e895c29e03)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.475014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.477412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 130/255] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:30 2022] GET /identity/v3/users/bc94a478ebd64d63b14964e895c29e03 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:30.482846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:30.594332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:30.629818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15306cc-7eb0-42e3-900e-1575e82d6af9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.678962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.679951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.680482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.680950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.682300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.683985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.686127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 126/256] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:30 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:30.692721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:30.703319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:30.706173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:30.883391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823e001-c2fa-4dd4-b1a6-8ff495f9d350 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.916177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640827e-2614-4c94-bf23-e0ba5384c5af None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:30.941639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.942204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.942473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.942738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.962207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:30.963659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:30.963938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:30.964207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/users/bc94a478ebd64d63b14964e895c29e03/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:30.968325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, user_id=bc94a478ebd64d63b14964e895c29e03, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:30.969765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:30.976186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:30.984033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:30.985366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:30.986756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 127/257] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:30.988600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.000717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.009803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 131/258] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:30 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/users/bc94a478ebd64d63b14964e895c29e03/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:31.020526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.134914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.176860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51029b52-5225-4a7d-a2f6-87962462791c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.216202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda621de-799d-44be-a50c-474597918e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.217117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda621de-799d-44be-a50c-474597918e64 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.217402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda621de-799d-44be-a50c-474597918e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.217677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda621de-799d-44be-a50c-474597918e64 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.218356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda621de-799d-44be-a50c-474597918e64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:31.218656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda621de-799d-44be-a50c-474597918e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:31.220050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda621de-799d-44be-a50c-474597918e64 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:31.226823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec262a6-fae0-496f-8d5d-91638176b326 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.242780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 132/259] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:31.248558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda621de-799d-44be-a50c-474597918e64 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.284489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.285141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.285420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.285694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.306288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:31.320977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:31.321989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:31.322782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 128/260] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:31.332104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.406165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.446780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90133-e96f-4bfa-97f4-c591eb139297 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.453375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda621de-799d-44be-a50c-474597918e64 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.482958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.483498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.483779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.484050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.484742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:31.485050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:31.487401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:31.496473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 129/261] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:31.502343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.555915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.587463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.588118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.588495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.588793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.593002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:31.594455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:31.596310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 133/262] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:31.603502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.608236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149681b0-e243-42ba-989f-b22d87afb277 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.661191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.672325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.673106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.673954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.674288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] PATH_INFO: `/v3/users/4fb8c9b617d44dc797a1d24915c1c505` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.678861 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb8c9b617d44dc797a1d24915c1c505)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:31.680376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:31.681972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 130/263] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/users/4fb8c9b617d44dc797a1d24915c1c505 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:31.687640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:31.811125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f381ef-c321-49b2-965d-2eca6a8c43a3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:31.851671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:31.852319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:31.852601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:31.852868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:31.924763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:31.932950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:31.934689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:31.990268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6ebf-2d3e-4cfa-91df-ca161ecc8e8e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:32.051294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 131/264] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:15:32.056572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:32.066816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:32.067746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:32.068181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:32.068727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:32.166631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:32.177124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b1058-b143-43c7-b712-a56c4f3fc893 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:32.218857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:32.219640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:32.219923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:32.220201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:32.221544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:32.223355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:32.226033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 132/265] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:32 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:32.231867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:32.233904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:32.245703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:32.247540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:32.407788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dca5-9808-4f8e-97b2-fbb86c1b2590 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:32.510971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:32.512560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:32.513014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:32.513549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:32.514901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4fb8c9b617d44dc797a1d24915c1c505) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:32.515379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:32.520027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 134/266] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 917 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:32.521973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:32.528285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:32.532544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 133/267] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:32 2022] GET /identity/v3/role_assignments?user.id=4fb8c9b617d44dc797a1d24915c1c505&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:32.675344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:32.680776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fac1e-b385-4c14-b0b7-02c9a892c7c1 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:32.741556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:32.742201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:32.742479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:32.742752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:32.743953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:32.745772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664b75b-80b4-46ee-9267-994c844c4bf5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:32.747757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 135/268] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:32 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:32.754603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:32.808583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:32.926413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc809e1-c684-41de-95a8-1262d18b1b3c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:32.981750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:32.982674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:32.983129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:32.983656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:32.984748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4fb8c9b617d44dc797a1d24915c1c505) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:32.985322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:32.987280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8723b-c7dc-48e9-a3a6-c8c13d7ac4a1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:32.997714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 134/269] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:32 2022] GET /identity/v3/role_assignments?user.id=4fb8c9b617d44dc797a1d24915c1c505&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:33.011000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151101-3467-4de8-972b-3d57f0de5f2a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.011283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151101-3467-4de8-972b-3d57f0de5f2a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.011563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4151101-3467-4de8-972b-3d57f0de5f2a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.013655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 136/270] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:33.019979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.020310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.020562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.042328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:33.043635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:33.109331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d9f61-b1f2-40ec-8683-b4370fb35ff4 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.110017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d9f61-b1f2-40ec-8683-b4370fb35ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.110017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d9f61-b1f2-40ec-8683-b4370fb35ff4 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.111794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 137/271] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:33.120687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.121200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.121564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.150621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 135/272] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:33.151592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:33.152956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:33.244521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 138/273] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:33.539156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:33.559259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:33.603197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:33.656873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f703e-3df8-4911-ad70-52fd19aa35f2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:33.702232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:33.703247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.703529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.703801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.705293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:33.708834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044aff7-3d65-4057-9ffa-97e3f5908917 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:33.709360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 136/274] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:33.710717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7c33-9e2d-489a-a45c-4d43a23c259d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.712623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7c33-9e2d-489a-a45c-4d43a23c259d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.712911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7c33-9e2d-489a-a45c-4d43a23c259d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.714352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 137/275] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:33.717114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7c33-9e2d-489a-a45c-4d43a23c259d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:33.723365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:33.735203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ffed9-6e2a-4190-b64e-156bc4385e33 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:33.773488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:33.774124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.774402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.774677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.785480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-73eed5cd-b16a-44e6-806a-4c096b1ebece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:33.790376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:33.791251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:33.792133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 139/276] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:33.792865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.792865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.793731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.814891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:33.816193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:33.819409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:33.835609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7c33-9e2d-489a-a45c-4d43a23c259d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:33.870113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:33.870724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:33.871001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:33.871274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:33.871931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:33.873398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:33.904285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 138/277] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:15:33 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:33.904285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:33.908085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 140/278] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:34.020875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a774f88-e4b6-4fd3-bca9-66bee3669e7e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.060443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.060977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.061255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.061582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.062254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:34.062553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:34.064502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:34.074075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 139/279] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:34.080573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.180954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:34.197840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2649e3-84b1-4d7d-873f-509971dc9c65 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.198675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.204483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:34.242508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.243206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.243488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.245150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] PATH_INFO: `/v3/users/6ccb56b7244443d880a6339d138aef99` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.262694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] RBAC: Authorizing `identity:get_user(user_id=6ccb56b7244443d880a6339d138aef99)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:34.264676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:34.266600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 141/280] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/users/6ccb56b7244443d880a6339d138aef99 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:34.273276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.370591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efd28-9d89-458f-b05b-e7d6aa96b850 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.411705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c455f43-1948-474d-9498-30be407e2620 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.425095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.426222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.426779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.427290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.440495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5ed64c79-ab49-4f89-864a-204522f2b7d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:34.446262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:34.447157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:34.447949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:34.450646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 140/281] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:34.452162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f1fc-4630-45d6-a40e-ff7d3e551453 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.452664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f1fc-4630-45d6-a40e-ff7d3e551453 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.453164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f1fc-4630-45d6-a40e-ff7d3e551453 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.454926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 141/282] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:34.456327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.456707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f1fc-4630-45d6-a40e-ff7d3e551453 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.456943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.457218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.457555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.458400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, user.id=6ccb56b7244443d880a6339d138aef99, scope.system=all) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:34.458708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:34.460160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66cf29-bea3-4a7c-b585-55f1f657d1b5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:34.470579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 142/283] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/role_assignments?user.id=6ccb56b7244443d880a6339d138aef99&scope.system=all&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:34.471978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.472285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.472559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.493250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:34.494549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:34.497052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:34.568374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:34.614291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f1fc-4630-45d6-a40e-ff7d3e551453 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.630373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 143/284] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:34.656114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.657135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.657678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.658930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.660038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:34.660601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:34.662259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:34.672790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 142/285] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:34.680309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.797835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28659e8-e683-4097-97ce-cbac2143b478 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:34.839126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:34.839757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:34.840331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:34.840599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] PATH_INFO: `/v3/users/4a868d5135a348c7963cbe3093d4f9ec` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:34.841737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] RBAC: Authorizing `identity:get_user(user_id=4a868d5135a348c7963cbe3093d4f9ec)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:34.843174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:34.845081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 144/286] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/users/4a868d5135a348c7963cbe3093d4f9ec => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:34.851060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:34.991501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6464b-8f2f-4833-9bdd-829843ec68c6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.027536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:35.039451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:35.039624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.040192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.040494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.040767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.041909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:35.043383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:35.045334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 143/287] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:34 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:35.051488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:35.061966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:35.203131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3ba02-b49f-4498-b717-7386a5015fc4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.216586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96d767-8945-4c6c-a918-843d23190ecf None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.272586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.273172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.273449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.273719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4a868d5135a348c7963cbe3093d4f9ec/roles/906b85ca0f0244f2ada233c8899f62f6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.276890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, user_id=4a868d5135a348c7963cbe3093d4f9ec, role_id=906b85ca0f0244f2ada233c8899f62f6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:35.278339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:35.281957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.282592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.283659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.284144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.296721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:35.306498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 144/288] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:15:35 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/users/4a868d5135a348c7963cbe3093d4f9ec/roles/906b85ca0f0244f2ada233c8899f62f6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:35.309699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:35.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:35.311530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 145/289] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:35.319933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:35.381616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:35.435973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddf7c4-0dd8-4c89-9975-d0884a4e17e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.480172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.480803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.481076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.481344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.481999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:35.482296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:35.483657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:35.496745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 145/290] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:35.502321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:35.583348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:35.626834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e3741-4831-456a-a4b3-3769fb82add6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.675967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.676761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.676961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.677281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] PATH_INFO: `/v3/users/bc94a478ebd64d63b14964e895c29e03` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.678487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] RBAC: Authorizing `identity:get_user(user_id=bc94a478ebd64d63b14964e895c29e03)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:35.679952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:35.681621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 146/291] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:35 2022] GET /identity/v3/users/bc94a478ebd64d63b14964e895c29e03 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:35.689164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:35.814487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d204-c44a-46ec-909a-62074f734532 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:35.838509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:35.862961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:35.863576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:35.863910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:35.864189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:35.961872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:35.963332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:36.104202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 146/292] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:36.109727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:36.235534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63357438-c143-4588-a4c2-bccf4edab137 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:36.273335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:36.274025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.274310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.274586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:36.275726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:36.277186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bf16f-8754-4830-9254-463e9c9ecddc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:36.279176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 147/293] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:36 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:36.284806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:36.320620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:36.467171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9dd63e-7afc-48fb-9878-d5129c311958 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:36.525905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:36.526521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.526803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.527076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:36.527805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=bc94a478ebd64d63b14964e895c29e03) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:36.528115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:36.529601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e03cb-4430-4662-a3da-3b6ac36617b4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:36.541020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 147/294] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:36 2022] GET /identity/v3/role_assignments?user.id=bc94a478ebd64d63b14964e895c29e03&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:36.823934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3493b-cf96-4159-a037-5a375706949e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.824456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3493b-cf96-4159-a037-5a375706949e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.824456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3493b-cf96-4159-a037-5a375706949e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:36.826048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 148/295] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:36.832719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ee11-aaf2-41bd-afd6-ab1e03bfdcfb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.833265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ee11-aaf2-41bd-afd6-ab1e03bfdcfb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.833265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ee11-aaf2-41bd-afd6-ab1e03bfdcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:36.853130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5ad7ee11-aaf2-41bd-afd6-ab1e03bfdcfb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:36.854351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5ad7ee11-aaf2-41bd-afd6-ab1e03bfdcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:36.928766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 148/296] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:36.975581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2034-c932-481f-baa1-8c9d225c208a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.975897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2034-c932-481f-baa1-8c9d225c208a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.976124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2034-c932-481f-baa1-8c9d225c208a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:36.977635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 149/297] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:36.982780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492310a7-bdc4-43e1-b1b7-afcf462b323b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:36.983062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492310a7-bdc4-43e1-b1b7-afcf462b323b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:36.983336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492310a7-bdc4-43e1-b1b7-afcf462b323b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.003038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-492310a7-bdc4-43e1-b1b7-afcf462b323b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:37.004350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-492310a7-bdc4-43e1-b1b7-afcf462b323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:37.007877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.084160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 149/298] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:37.158742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f488-0be0-4a7c-bd3d-204c276a6a92 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.159082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f488-0be0-4a7c-bd3d-204c276a6a92 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.159288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f488-0be0-4a7c-bd3d-204c276a6a92 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.160917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 150/299] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:37.165761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.166047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.166318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.185997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:37.187160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:37.222065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f488-0be0-4a7c-bd3d-204c276a6a92 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.226100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.270291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 150/300] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:37.349322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f488-0be0-4a7c-bd3d-204c276a6a92 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.374804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.384870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.385502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.385776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.386051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.396850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:37.402345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:37.403237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:37.404038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 151/301] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:37.411355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.467432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.483743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.513488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cab360-0196-4c59-9707-85dc3dcea321 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.527880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22b2e-b107-4523-b120-5570342c5942 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.559187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.559816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.560107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.560408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.561497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:37.562920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:37.565417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 151/302] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:37.566177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.569080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.569927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.570198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.570467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.571131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:37.571432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:37.572828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:37.581723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 152/303] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:37.582395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.670160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e788-c6fc-4baa-b449-abf1828e7255 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.701875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a926309-7428-46ee-88eb-a78c5a15e17e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.707126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.707755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.708028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.708315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.716559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.721174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:37.726700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:37.727580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 152/304] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:37.733502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.737233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:37.748879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.749492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.749765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.750030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] PATH_INFO: `/v3/groups` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.750651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:37.752664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:37.768622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 153/305] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Sep 26 05:15:37 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:15:37.769445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.849215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77775-c9f5-47b5-8c0a-d6d69c251993 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.902480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.903387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.903678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.903954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] PATH_INFO: `/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.908747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d85d-29e8-4144-b4a5-382c4a49f262 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:37.925346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] RBAC: Authorizing `identity:get_user(user_id=9127e0f5f36e40f7bde9a71f1494b0bc)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:37.927130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:37.932431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 153/306] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:37.935793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:37.973717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:37.974351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:37.974632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:37.974907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:37.975595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:37.975896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:37.977460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:37.990647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 154/307] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:37.997360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:38.016657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:38.055211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:38.074136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d2219-28cb-4504-b5d3-31771ce372e3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:38.119000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92543ef-312c-4024-bee1-1cce421d7b60 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:38.119236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:38.120070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.120395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.120672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.121399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, user.id=9127e0f5f36e40f7bde9a71f1494b0bc, scope.system=all) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:38.121710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:38.123138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00645084-e7a3-4e47-9b75-e5b58375638c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:38.136670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 154/308] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/role_assignments?user.id=9127e0f5f36e40f7bde9a71f1494b0bc&scope.system=all&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:38.139019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d567f-087b-424c-bb77-f744829c52a4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.139578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d567f-087b-424c-bb77-f744829c52a4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.140052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d567f-087b-424c-bb77-f744829c52a4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.148890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 155/309] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:38.153025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.154546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.155579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.163705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:38.164387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.164666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.164938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] PATH_INFO: `/v3/users/6ccb56b7244443d880a6339d138aef99` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.166595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] RBAC: Authorizing `identity:get_user(user_id=6ccb56b7244443d880a6339d138aef99)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:38.168105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:38.174929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 155/310] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:37 2022] GET /identity/v3/users/6ccb56b7244443d880a6339d138aef99 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:38.175995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:38.190942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:38.193495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:38.287566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 156/311] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:38.310677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce345c-f7bd-428c-b7fd-80e0e87c1278 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:38.325343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:38.347297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:38.347955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.348242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.348548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] PATH_INFO: `/v3/system/users/6ccb56b7244443d880a6339d138aef99/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.350119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ccb56b7244443d880a6339d138aef99, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:38.351484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:38.363565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 156/312] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Sep 26 05:15:38 2022] PUT /identity/v3/system/users/6ccb56b7244443d880a6339d138aef99/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:38.561237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:38.613884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:38.666767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394a3fa-e549-4dd8-80b9-d13a535c986f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:38.702336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:38.702973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.703250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.703516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.714337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:38.719198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:38.720648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:38.721939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 157/313] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:38.727120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:38.833780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ec088-4571-4527-a93d-62778d8e796e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:38.868361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:38.868975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:38.869253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:38.869522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:38.870446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:38.870745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:38.872122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:38.881669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 157/314] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:38.887276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:38.995524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331f2c3-8a61-4e5c-b123-2bb422bbba57 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:39.006978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:39.032330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:39.033609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:39.033889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:39.034213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] PATH_INFO: `/v3/users/4a868d5135a348c7963cbe3093d4f9ec` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:39.035362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] RBAC: Authorizing `identity:get_user(user_id=4a868d5135a348c7963cbe3093d4f9ec)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:39.036858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:39.038737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 158/315] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:38 2022] GET /identity/v3/users/4a868d5135a348c7963cbe3093d4f9ec => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:39.044288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:39.092790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:39.154081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c023f-f0b6-4bce-af5b-d37136056fbf None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:39.189568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:39.190248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:39.190531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:39.190802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:39.260586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:39.262097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:39.304112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:39.365762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 158/316] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:39.371416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:39.484283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9433bb-93bf-4994-b9b8-ccc3877d21b1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:39.523690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:39.524355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:39.524637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:39.524913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:39.526042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:39.527499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc025-12c2-4d90-8bd3-e7cafaf67ab2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:39.529311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 159/317] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:39 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:39.535534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:39.640567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5f865-9913-48fe-bf4c-f5732a5aa82a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:39.676771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:39.677377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:39.677655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:39.677929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:39.678654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, scope.project.id=267a872030254d2c8a0328ffe37ea732, user.id=4a868d5135a348c7963cbe3093d4f9ec) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:39.678961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:39.680487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfd5b4-206c-48c7-a2ff-c01f0cb8b8eb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:39.686874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 159/318] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:15:39 2022] GET /identity/v3/role_assignments?user.id=4a868d5135a348c7963cbe3093d4f9ec&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:40.095635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c717ee-e94c-4e55-8e6f-4452c15ed4ea None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.096146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c717ee-e94c-4e55-8e6f-4452c15ed4ea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.096326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c717ee-e94c-4e55-8e6f-4452c15ed4ea None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.097941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 160/319] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:40.103489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360302b-4c45-4c22-80b4-31cb4da04a8a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.103952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360302b-4c45-4c22-80b4-31cb4da04a8a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.104101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360302b-4c45-4c22-80b4-31cb4da04a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.111817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:40.123882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b360302b-4c45-4c22-80b4-31cb4da04a8a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:40.125090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b360302b-4c45-4c22-80b4-31cb4da04a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:40.196334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 160/320] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:40.427231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0306-18bd-4713-8c2a-5debc7d5d317 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.427410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0306-18bd-4713-8c2a-5debc7d5d317 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.427685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0306-18bd-4713-8c2a-5debc7d5d317 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.429217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 161/321] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:40.434531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.434815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.435086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.453691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:40.455157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:40.460596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0306-18bd-4713-8c2a-5debc7d5d317 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:40.516515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:40.532959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 161/322] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:40.577984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0306-18bd-4713-8c2a-5debc7d5d317 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:40.613404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:40.614037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.614315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.614588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.615712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:40.617199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eea9f9-f5ee-4594-bb47-138e3e8f3089 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:40.618997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 162/323] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:15:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:40.624576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:40.635755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564142d5-74fc-48be-af31-0067da9260a4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.636551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564142d5-74fc-48be-af31-0067da9260a4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.636827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564142d5-74fc-48be-af31-0067da9260a4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.638472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 163/324] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:40.644085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.644390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.644769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.663272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:40.664492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:40.693017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:40.734437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 164/325] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:40.735080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68a69d-f950-46fc-8582-6ee743437dbd None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:40.770595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:40.771234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.771513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.771781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] PATH_INFO: `/v3/groups` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.772481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:40.774104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:40.796271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 162/326] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Sep 26 05:15:40 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:15:40.806227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:40.859073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:40.922694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7899-f23e-4884-b3a3-1c7dd386f1b7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:40.960024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:40.960720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:40.961048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:40.961325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:40.975451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:40.980379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:40.981246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:40.982037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 165/327] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:40.988276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.044333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.112895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab914d-bbea-48e3-948a-6539007a54c9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.145089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:41.161139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.162097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.162378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.162654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.163624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:41.163624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.165077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.175436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 163/328] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:41.181766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.198451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d25c8-5411-4b8b-9f17-4fcd9233dfbd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.202149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:41.246666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.247228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.247507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.247775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.262711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:41.268980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:41.269875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:41.270665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 166/329] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:41.276357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.322114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353fb79-4d1d-442a-b44b-7bf9867df98f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.366380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.366997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.367270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.367539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] PATH_INFO: `/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.369258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] RBAC: Authorizing `identity:get_user(user_id=9127e0f5f36e40f7bde9a71f1494b0bc)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.370960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.378945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 164/330] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:41 2022] GET /identity/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:41.380840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.405365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0bcec-881d-4975-b58e-79f3f3b77036 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.449040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.450030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.450374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.450843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.451683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:41.452167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.452931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:41.454519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.464375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 167/331] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:41.471309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.515101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1f8c-8cd9-46f0-95a9-cb3ee4200b2f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.519567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:41.551301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.552298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.552753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.553224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] PATH_INFO: `/v3/system/users/9127e0f5f36e40f7bde9a71f1494b0bc/roles/906b85ca0f0244f2ada233c8899f62f6` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.554842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9127e0f5f36e40f7bde9a71f1494b0bc, role_id=906b85ca0f0244f2ada233c8899f62f6)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.556520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.568006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 165/332] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Sep 26 05:15:41 2022] PUT /identity/v3/system/users/9127e0f5f36e40f7bde9a71f1494b0bc/roles/906b85ca0f0244f2ada233c8899f62f6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:41.611816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b763e-0766-499c-b2a5-d113475d3824 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.659743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.660510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.660790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.661058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] PATH_INFO: `/v3/users/6ccb56b7244443d880a6339d138aef99` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.662250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] RBAC: Authorizing `identity:get_user(user_id=6ccb56b7244443d880a6339d138aef99)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.663721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38dc27-89c9-4efa-ba79-f805ad95f463 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.665850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 168/333] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:41 2022] GET /identity/v3/users/6ccb56b7244443d880a6339d138aef99 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:41.673628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:41.805388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d5b7-d056-49f2-8775-7cc165c5fa35 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:41.824986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:41.843863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:41.844509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:41.844786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:41.845087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:41.845903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] RBAC: Adding query filter params (role.id=1e1b8d5397754a2e93980b3202ff7e7f, user.id=6ccb56b7244443d880a6339d138aef99, scope.system=all) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:41.846214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:41.847649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0beb76-9cce-45ae-b403-8c8895573d91 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:41.861888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 166/334] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 05:15:41 2022] GET /identity/v3/role_assignments?user.id=6ccb56b7244443d880a6339d138aef99&scope.system=all&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:43.792628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12c29-6c20-45c0-9bee-4fbf631e8f81 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:43.792628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12c29-6c20-45c0-9bee-4fbf631e8f81 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:43.792628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b12c29-6c20-45c0-9bee-4fbf631e8f81 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:43.793697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 169/335] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:43.799443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:43.800308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:43.800806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:43.822842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f916e-4e0e-42d5-8f20-160c425c5743 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:43.823438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f916e-4e0e-42d5-8f20-160c425c5743 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:43.823902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f916e-4e0e-42d5-8f20-160c425c5743 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:43.825444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 170/336] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:43.826517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:43.827919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:43.834337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb238-d691-46b6-a7dc-c54acbf7b375 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:43.834888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb238-d691-46b6-a7dc-c54acbf7b375 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:43.835354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb238-d691-46b6-a7dc-c54acbf7b375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:43.864108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9ffeb238-d691-46b6-a7dc-c54acbf7b375 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:43.865668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9ffeb238-d691-46b6-a7dc-c54acbf7b375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:43.972657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 167/337] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:44.012398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 171/338] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:44.260516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:44.344905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70268673-dc5b-4b39-8d5f-ecab82ac72fb None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.346097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70268673-dc5b-4b39-8d5f-ecab82ac72fb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.346564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70268673-dc5b-4b39-8d5f-ecab82ac72fb None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.352202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 172/339] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:44.353824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30606-3f88-478d-8678-43a12307deeb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.354310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30606-3f88-478d-8678-43a12307deeb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.354767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30606-3f88-478d-8678-43a12307deeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.374877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403933f-422e-4291-ad1b-7545d2a4a7c9 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:44.382701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:44.386260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-92c30606-3f88-478d-8678-43a12307deeb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:44.387717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-92c30606-3f88-478d-8678-43a12307deeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:44.415832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:44.416590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:44.418094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.418592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.419098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.431480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:44.436757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:44.437638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:44.438419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:44.441432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 168/340] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:44.447361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:44.498394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 173/341] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:44.555335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66153a5-eb90-4e74-b559-956236b773dc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:44.569438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30606-3f88-478d-8678-43a12307deeb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:44.592511 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:44.593376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.593796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.594193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.595255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:44.595799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:44.597349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:44.606942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 169/342] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:44.613112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:44.651563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:44.749652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8816e-3ac3-48d2-8ef2-496bdd32d3c0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:44.759349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30606-3f88-478d-8678-43a12307deeb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:44.791646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:44.792536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.792959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.793373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.809877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:44.810520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.810802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.811078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.822220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5aabadb9-b797-45e8-82a3-9463575ed7bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:44.827418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:44.828504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:44.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 174/343] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:44.830588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4974b-8acb-4a5d-9293-97675a619667 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:44.830820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4974b-8acb-4a5d-9293-97675a619667 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:44.831097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4974b-8acb-4a5d-9293-97675a619667 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:44.837284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 175/344] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:44.838242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4974b-8acb-4a5d-9293-97675a619667 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:44.856829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:44.883733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:44.886260 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe2d3d-e5ea-4ce1-8e6d-159fb569926c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:44.917688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:44.962994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4974b-8acb-4a5d-9293-97675a619667 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.006335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.006335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.006335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.006335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.006742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:45.006742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.006961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 170/345] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:15:45.008317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.008583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.009021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.009430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.019431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 176/346] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:45.024530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.034787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:45.036745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:45.115705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 171/347] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:45.116600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.144904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.161275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65080c1-1655-47a5-8a72-6bae0363c156 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.183063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.201223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.202372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.202851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.203351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.205494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.207298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.209805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 177/348] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:45.211499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.235584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c9c72-a7de-4819-b4cf-168e9eb731ee None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.275246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.275947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.276187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.276497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.289073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:45.295141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:45.296091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 172/349] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:45.298939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.356346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e26d2-7cd1-4bbc-a85d-403d7080bb93 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.428261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef4211-a68d-4b8f-8d0e-e0988488e3b5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.446336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.447290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.447794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.448303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.449739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.472569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.473205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.473481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.473754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.474873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.482177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.483907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.487516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 173/350] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:45.492954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.570544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.572169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.624843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15794a2-0783-4c2e-94f5-4eedb15ce7b9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.673248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.674174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.674310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.674488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.675159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:45.675549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.676875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.686935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 174/351] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:45.688019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.696189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 178/352] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:45.697363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.746562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.757724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:45.819226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7beb1-4585-46d3-9ba1-c300399a22ea None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.824072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191be0a-01d7-46bd-9ae8-5d57f7766048 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:45.858197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.858806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.859087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.859423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.860088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:45.860969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:45.862518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754c9d-d3e2-40c6-9086-d269ea57f494 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:45.862875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:45.863507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.863783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.864049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.869095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 175/353] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:45.870635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda081d-6295-4313-93ac-c0619fee6784 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:45.870925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda081d-6295-4313-93ac-c0619fee6784 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:45.871199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda081d-6295-4313-93ac-c0619fee6784 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:45.872728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 176/354] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:45.873456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda081d-6295-4313-93ac-c0619fee6784 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:45.888858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:45.896039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:45.897317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 179/355] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:45.898292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.030489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda081d-6295-4313-93ac-c0619fee6784 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.042538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eca8da-c1d1-45d0-b813-c616e56ccceb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.044732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.046026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.090079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.090727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.092258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.092557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.093943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.095937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278591c0-1c10-4d96-be73-d1e643b21244 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.100345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 180/356] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.101231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.101574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.101857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.102132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.102682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.103133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.103609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] PATH_INFO: `/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.105421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] RBAC: Authorizing `identity:get_user(user_id=9127e0f5f36e40f7bde9a71f1494b0bc)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.106838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.112372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 177/357] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:15:45 2022] GET /identity/v3/users/9127e0f5f36e40f7bde9a71f1494b0bc => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.113341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.135752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:46.137048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:46.216146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 181/358] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:46.217101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.270033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f797a-8e1c-4e07-b9b7-e92e12a4ad2a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.313951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.315472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.315971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.316467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.317817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:46.318338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.320613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.332597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 178/359] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.333916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.342514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b4728-ab3e-4627-a463-2b3bd2b42c27 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.344236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.386045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.386763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.387044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.387315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.388547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.390063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.392934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 182/360] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.393160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.394626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.473850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4444a-a579-45ae-b3d1-a23c438c1ef3 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.516394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.518032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.518528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.519252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.520678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] RBAC: Adding query filter params (role.id=906b85ca0f0244f2ada233c8899f62f6, user.id=9127e0f5f36e40f7bde9a71f1494b0bc, scope.system=all) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:46.521210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.523251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.526874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca51e-30f1-478f-9dc4-938c787c84d9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.539481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 179/361] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/role_assignments?user.id=9127e0f5f36e40f7bde9a71f1494b0bc&scope.system=all&role.id=906b85ca0f0244f2ada233c8899f62f6 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.541116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.571686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.572224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.572530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.572803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.620789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.655998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.658191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68eb891-f69e-4d81-ac57-4deb2373d352 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.669606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.677809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883fa17-c6fd-4580-a475-f36f6d103b48 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.723963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:46.726436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.727247 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.727742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.728984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:46.729879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:46.732006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6ad69-56da-453b-b80c-758ff0efa690 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:46.741944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 180/362] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:46.743668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2465d-226a-4c3b-b493-e229e2f17ab7 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.744172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2465d-226a-4c3b-b493-e229e2f17ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.749626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2465d-226a-4c3b-b493-e229e2f17ab7 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.752360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 181/363] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:46.753932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2465d-226a-4c3b-b493-e229e2f17ab7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:46.810808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 183/364] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:46.814063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:46.814348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:46.814642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:46.849457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:46.850607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:46.950703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:46.954071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2465d-226a-4c3b-b493-e229e2f17ab7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:46.955565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 184/365] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:46.956564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.003138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.004183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.004707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.005185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.021742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:47.026927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:47.027816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:47.028702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:47.032951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 182/366] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:47.037602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.069503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:47.096808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e11c7-cdca-4f42-bd4a-4eebdd412a1f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.177583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.178229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.178510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.178781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.179853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c9fb0-0af3-43a3-acbe-f25a9a198cd3 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.184982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.186447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.189001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 185/367] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:46 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:47.194644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.223066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.224094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.224596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.225079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.226555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:47.227101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.228938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.240132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 183/368] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:47.246138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.336853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:47.372634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:47.379480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6aa22-4750-49f0-b2f6-957b4faa7af6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.392585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291917dd-b514-467f-bc97-3cf340234326 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.428983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.429682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.430030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.430370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.431805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.433101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.433719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.434413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.437173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.440480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.443881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 186/369] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:47.445037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.515341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.516850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.593331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33bd9-79be-497a-8a85-0066407a92df None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.620690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:47.645323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 184/370] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:47.646192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.648162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.650323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.650816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.651280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.654899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:47.669061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:47.677037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:47.677945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:47.678754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:47.682100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 187/371] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:47.683111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.780596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e35d5-3c5d-44ec-a99a-ac11d4971db5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.827107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89abe-93d4-4077-ac38-21d9ad0026ca None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:47.828121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.828841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.829660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.830156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.831241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=267a872030254d2c8a0328ffe37ea732) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:47.831555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.833095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.841200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 185/372] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:47.842090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.876979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:47.877664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:47.877941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:47.878215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:47.879545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:47.881027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:47.883811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 188/373] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:47.890283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:47.912723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:48.005157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd1c6f-536a-432e-a09d-ba1e351054dc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.036379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:48.081855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.081855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.082455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.082455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.083013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:48.083316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.084730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.093675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 186/374] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:48.101961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:48.118474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03a3a-c138-4d37-9ac1-d29c35e74e2e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.181082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.181637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.181907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.182171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.183399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.187789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.191026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 189/375] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:47 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:48.197284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:48.273540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bd21f-23f2-411b-9b2f-c9b1bd85e69f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.285518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:48.316599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.317336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.317615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.317889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.398107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.399569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.415914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0668831-355d-426b-a713-5efa900c2d7f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.493299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:48.502637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.508974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.509257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.509535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.510278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=267a872030254d2c8a0328ffe37ea732) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:48.510588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.512026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.514151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 187/376] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:48.520507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:48.523303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 190/377] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:48 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 260 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:48.531475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:48.640134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781d34-29ef-453b-8c72-256e76a84d41 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.682711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.683361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.683642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.683917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] PATH_INFO: `/v3/projects/3af173ac0b7d40069de1fa09770699a3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.685255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] RBAC: Authorizing `identity:get_project(project_id=3af173ac0b7d40069de1fa09770699a3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.686623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ac1e9-e3e5-4e1c-88e9-ca5696513d7e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.690191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 188/378] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:48 2022] GET /identity/v3/projects/3af173ac0b7d40069de1fa09770699a3 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:48.694714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:48.787439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64545-80dd-4883-8c3a-a952db605e30 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.787610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64545-80dd-4883-8c3a-a952db605e30 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.787925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64545-80dd-4883-8c3a-a952db605e30 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.789704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 189/379] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:48.797268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.797594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.797818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.819770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:48.821125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:48.834988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e176d9-1c06-4be4-baff-6f9f03d27792 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:48.890391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:48.891003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:48.891289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:48.892131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] PATH_INFO: `/v3/groups/3ad883a183f145ce9f8560850fd88e48` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:48.898250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] RBAC: Authorizing `identity:get_group(group_id=3ad883a183f145ce9f8560850fd88e48)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:48.900004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:48.901723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 191/381] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:48 2022] GET /identity/v3/groups/3ad883a183f145ce9f8560850fd88e48 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:48.906371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 190/381] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:48.908444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:48.973056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:49.019896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da69b8a-5d8d-482c-ab3a-f83ca3db075c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:49.057095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:49.057732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.058017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.058286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.059079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] RBAC: Adding query filter params (group.id=3ad883a183f145ce9f8560850fd88e48, role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=3af173ac0b7d40069de1fa09770699a3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:49.059388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:49.060871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:49.069979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 192/382] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:48 2022] GET /identity/v3/role_assignments?group.id=3ad883a183f145ce9f8560850fd88e48&scope.project.id=3af173ac0b7d40069de1fa09770699a3&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:49.213455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:49.223089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:49.334121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c813c-27d1-406a-a0e2-8f5962fc79fe None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:49.372518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:49.373158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.373436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.373703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.387051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:49.392819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:49.393768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:49.394607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 191/383] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:49.401483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:49.549414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2824b-5d19-4eb5-ad92-14f41641582e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:49.584362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:49.596169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:49.596876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.597161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.597435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.598101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] RBAC: Adding query filter params (name=member) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:49.598405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:49.599789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473edf2-8c1d-4de0-9051-90c9170ee59c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:49.609026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 193/384] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:49.615920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:49.717133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:49.717441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea84c8-be20-45ac-9d6b-6bce4b967297 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.717824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea84c8-be20-45ac-9d6b-6bce4b967297 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.718203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea84c8-be20-45ac-9d6b-6bce4b967297 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.720506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 194/385] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:49.725906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.726304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.726685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.756751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954595e-91b2-45eb-a807-55eb34f9fa44 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:49.765258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:49.766615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:49.795415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:49.796050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.796357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.796623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.797963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:49.799661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:49.801468 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 192/386] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:49 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:49.807156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:49.886852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 195/387] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:49.925500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610d88-d3e0-43bc-9b6e-868d660a9f7c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:49.969290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:49.970513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:49.970786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:49.971059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:49.972240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:49.973809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:49.975806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 193/388] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:49 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:49.982495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:50.099797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68320701-470c-4d29-9d7c-ff3a3105fb66 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:50.127397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:50.150391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:50.150980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:50.151264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:50.151539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/groups/d947b3a27d2242e78e6588ff42521051/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:50.153493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, group_id=d947b3a27d2242e78e6588ff42521051, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:50.155089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:50.186388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 196/389] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 05:15:49 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/groups/d947b3a27d2242e78e6588ff42521051/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:50.334768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:50.337922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:50.461349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb593-1ebd-42eb-9501-d9f8db770bec None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:50.508083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:50.508757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:50.509031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:50.509300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:50.524746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:50.529627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:50.530575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:50.531382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 194/390] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:50.539377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:50.665727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067386c-ba50-4508-b945-f238e4f2edb8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:50.705883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:50.706498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:50.706781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:50.707054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:50.707716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:50.708016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:50.709416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:50.724074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 197/391] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:50.729327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:50.810769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:50.863217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0ecf-d4bf-45f6-ba0e-cce9b207bf76 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:50.908374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:50.909024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:50.909299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:50.909568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:50.910840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:50.912364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d78fb-b9ae-41e2-a60d-cfaa5df2ee18 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:50.914405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 195/392] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:50 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:50.920553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:50.977397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:51.055131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d22f-b46d-438f-aa3b-23a3f92ffb1b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:51.115741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:51.116428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.116710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.116985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.120935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:51.122404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:51.124461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 198/393] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:50 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:51.126714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc03c-430e-444a-b58a-43c75c5c7c18 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.127006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc03c-430e-444a-b58a-43c75c5c7c18 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.127280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc03c-430e-444a-b58a-43c75c5c7c18 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.128621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 196/394] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:51.133447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:51.133716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.133995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.134267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.167576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:51.168815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:51.288441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 197/395] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:51.300624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25ce24-a85e-4c7a-a4a2-6586e33dbe86 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:51.307882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:51.345199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:51.345884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.346156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.346424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c/groups/d947b3a27d2242e78e6588ff42521051/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.348669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] RBAC: Authorizing `identity:create_grant(project_id=f415ede09edd4a15803a41d7f0ac862c, group_id=d947b3a27d2242e78e6588ff42521051, role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:51.350166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d0d1-e65b-4ec9-a2ac-8886ee096407 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:51.374488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 199/396] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 05:15:51 2022] PUT /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c/groups/d947b3a27d2242e78e6588ff42521051/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:51.637946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:51.709593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:51.741458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a527b52-c658-4da6-a601-22898cabbc2b None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.741753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a527b52-c658-4da6-a601-22898cabbc2b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.742038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a527b52-c658-4da6-a601-22898cabbc2b None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.743694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 200/397] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:51.758080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.758296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.758505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.775089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1d0a-1747-42e8-843f-63a5633664d7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:51.781327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:51.783365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:51.820797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:51.821446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:51.821727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:51.822007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:51.845827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:51.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:51.851655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:51.852899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 198/398] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:51.857806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:51.863795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 201/399] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:52.013750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9901b2-66cd-4923-9297-19530f529b21 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.040600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.064068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.064717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.064995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.065262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.065921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:52.066218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.067569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0cf60-6b84-4626-9440-d6c6326e4a95 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:52.076272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 199/400] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:52.084373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.099454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce86d3-2018-4112-b95f-ee29d070ff7f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.100122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce86d3-2018-4112-b95f-ee29d070ff7f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.100638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce86d3-2018-4112-b95f-ee29d070ff7f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.102484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 200/401] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:52.108772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.111884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.111884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.139941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:52.141167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:52.248139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 201/402] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:52.263363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.264727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.286363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b650f17-8b18-4839-a79e-f71b84023cd7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.355838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.356582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.356582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.358194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.359396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.362342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:52.365276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 202/403] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:52.369551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.436915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59f4e8-6357-4b2d-94ee-c9a22cc7ff16 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.464613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.508531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.509216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.509491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.509754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.522677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:52.528330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:52.529245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:52.530058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 202/404] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:52.537287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.559254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e975-58f6-4d1e-b378-0a72ae8b72ce None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.597630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.598375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.598721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.599056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.600408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.601977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:52.603776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 203/405] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:52.608288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.656102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.676013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20ade-9f98-4ac8-acca-11f4c99983d2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.713988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.714586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.714903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.715174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.715829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:52.716451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.717529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:52.731972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 203/406] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:52.732871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.749347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a788c-94b5-40b0-b091-a45d831110ec None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.771741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.796130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.796967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.797329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.797684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.809620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:52.815428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:52.816365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:52.817162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:52.821506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 204/407] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:52.822334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.863852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544aff-4540-4d27-82b6-45d5b3b4a572 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.911225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.912771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.913552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.914076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/groups/d947b3a27d2242e78e6588ff42521051/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.916858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, group_id=d947b3a27d2242e78e6588ff42521051, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.919196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:52.922084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:52.933355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5c5e-50d7-4211-82ad-b0224ceda1f7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:52.941787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 204/408] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 05:15:52 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/groups/d947b3a27d2242e78e6588ff42521051/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:52.948958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:52.991652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:52.992427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:52.992743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:52.993025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:52.994297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:52.997383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.002906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 205/409] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:53.008192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.167596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:53.178632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc64301-e772-4df8-b86f-df7e8b4640df None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.184376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0784-a85d-4262-87a4-e9e31980ea22 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.224436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.225437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.225914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.226340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.227267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.227785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.228071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.228495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.229150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.229868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:53.230235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.230447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe591c-08d3-40ec-b109-44233cd090d2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.231698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531bf7b-1a77-4a09-a836-e9723f53f9b0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.232412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 206/410] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:53 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:53.233401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166360-7f25-4df1-a5c0-33fd825258d6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.233680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166360-7f25-4df1-a5c0-33fd825258d6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.234245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166360-7f25-4df1-a5c0-33fd825258d6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.236150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 207/411] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:15:53.239217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36166360-7f25-4df1-a5c0-33fd825258d6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.246624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 205/412] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:53.247703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.247985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.248275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.272619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:53.274128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:53.279936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:53.359503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 206/413] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:15:53.360499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.387881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36166360-7f25-4df1-a5c0-33fd825258d6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.463579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:53.484986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.486250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.486528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.486795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732/groups/d947b3a27d2242e78e6588ff42521051/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.488451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810944b-94d5-416b-b594-057bd8db1672 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.492126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] RBAC: Authorizing `identity:create_grant(project_id=267a872030254d2c8a0328ffe37ea732, group_id=d947b3a27d2242e78e6588ff42521051, role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.494210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.530501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 208/414] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 05:15:53 2022] PUT /identity/v3/projects/267a872030254d2c8a0328ffe37ea732/groups/d947b3a27d2242e78e6588ff42521051/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:15:53.536457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.537103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.537381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.537650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] PATH_INFO: `/v3/groups/3ad883a183f145ce9f8560850fd88e48` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.538867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] RBAC: Authorizing `identity:get_group(group_id=3ad883a183f145ce9f8560850fd88e48)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.540433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.542646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 207/415] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:53 2022] GET /identity/v3/groups/3ad883a183f145ce9f8560850fd88e48 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:53.548602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.671639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:53.688459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f5fe-a734-468e-b8de-942b8d1e1a22 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.725845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.726577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.726903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.727304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] PATH_INFO: `/v3/projects/3af173ac0b7d40069de1fa09770699a3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.728645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] RBAC: Authorizing `identity:get_project(project_id=3af173ac0b7d40069de1fa09770699a3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.730103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.731765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 209/416] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:53 2022] GET /identity/v3/projects/3af173ac0b7d40069de1fa09770699a3 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:53.739358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.757326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:53.889982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4fd88-6667-41a6-a166-8661767c837e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.903586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98adc7-d33b-476c-aaf8-a57bc21bc846 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:53.936225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.936835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.937108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.937375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] PATH_INFO: `/v3/projects/3af173ac0b7d40069de1fa09770699a3/groups/3ad883a183f145ce9f8560850fd88e48/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.939639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=3af173ac0b7d40069de1fa09770699a3, group_id=3ad883a183f145ce9f8560850fd88e48, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:53.941110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:53.956880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 208/417] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 05:15:53 2022] PUT /identity/v3/projects/3af173ac0b7d40069de1fa09770699a3/groups/3ad883a183f145ce9f8560850fd88e48/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:15:53.968850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:53.969667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:53.970429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:53.970672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:53.991929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b6e7c74c-62ce-4a0d-805d-53f58173ee4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:53.997841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:53.999790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:54.000847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 210/418] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:54.005465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.026630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74eb3b-4ce7-4868-8991-7f6f63519f7e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.026951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74eb3b-4ce7-4868-8991-7f6f63519f7e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.027167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74eb3b-4ce7-4868-8991-7f6f63519f7e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.028027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.029079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 211/419] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:54.037072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.037400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.037678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.081093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:54.082292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:54.111452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.151554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cdf3-a628-4254-b59a-715185db9c5e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:54.195025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:54.195736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.196472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.198552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.199385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:54.199670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:54.201071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:54.212010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 209/420] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:54.218118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.224707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 212/421] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:54.303587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.337491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb978b8e-555e-4a05-8806-a894b1f77e38 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:54.371560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:54.372166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.374148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.374148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.439970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:54.441511 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:54.521029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.544916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 210/422] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:15:54.550316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.619899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.737820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8704bd0-48d8-4287-bbe2-92a1fd10f782 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:54.765525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eae1-00e2-4f46-acbd-93c98ae29eec None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:54.802113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:54.803064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.803563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.804030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.806258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:54.807910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:54.811341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 213/423] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:54.816409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.833873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:54.834838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:54.835470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:54.836668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:54.841032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.859683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:54.864805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:54.865696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:54.866516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 211/424] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:54.876956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:54.940184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:54.974307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e64d-c0f3-46ab-a224-578acaa83930 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.011443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:55.012431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.012906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.013446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.014746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:55.016486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:55.018549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 214/425] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:55.024510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:55.095077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eae339-0d9f-4330-ac38-fdde37f7fcee None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.137873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:55.165674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:55.166288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.166567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.166839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.167503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:55.167806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:55.172053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381236fe-1916-4755-bb69-9674409fc906 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.174453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:55.193162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 212/426] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:55.199168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:55.226143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:55.226797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.227033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.227310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.228038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:55.228419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:55.230098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:55.240758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 215/427] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:55 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:55.382087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:55.385880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0ffbb-1025-48a0-bac3-f0af486b6454 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.438970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:55.439604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.439957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.440228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.522079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:55.523532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdceae6-c972-458e-862f-cac7d7fe5645 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:55.643319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:55.648444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 213/428] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:55.655001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:55.712087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cf35-2487-4697-811a-9ed934977271 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.712423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cf35-2487-4697-811a-9ed934977271 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.712684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cf35-2487-4697-811a-9ed934977271 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.714371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 214/429] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:55.721456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.721863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.722238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.742704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:55.744099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:55.781642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc84d-d443-4a95-9494-a4fe593dfbe7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.823221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 215/430] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:55.828728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:55.829890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:55.830368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:55.830868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] PATH_INFO: `/v3/projects/f415ede09edd4a15803a41d7f0ac862c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:55.832195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] RBAC: Authorizing `identity:get_project(project_id=f415ede09edd4a15803a41d7f0ac862c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:55.833843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:55.835927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 216/431] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:55 2022] GET /identity/v3/projects/f415ede09edd4a15803a41d7f0ac862c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:55.841132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:55.968102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fad18-23eb-41e0-af28-da5f0fd0d600 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:55.996709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.009424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.010068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.010347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.010674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.011811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:56.013327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:56.015240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 216/432] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:55 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:56.021328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.045812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.159033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600f80a-0a10-4e03-b1b1-1cef339e76bf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:56.195343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.209643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.210539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.211794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.212445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.216289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=f415ede09edd4a15803a41d7f0ac862c) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:56.216614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:56.218444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d908b-32d8-4fbe-8740-2c682dbad129 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:56.226858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 217/433] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=f415ede09edd4a15803a41d7f0ac862c&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:56.328030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c640-baf4-4685-844a-359e1298d2bd None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:56.367746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.368791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.369279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.370031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.381179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dca2d-0dc8-4bdc-9d46-e4463cff58fd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.381479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dca2d-0dc8-4bdc-9d46-e4463cff58fd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.381736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dca2d-0dc8-4bdc-9d46-e4463cff58fd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.383408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 218/434] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:56.384574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:56.391134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2640-82f1-49e7-9747-6ec977355def None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.391532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2640-82f1-49e7-9747-6ec977355def None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.391738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2640-82f1-49e7-9747-6ec977355def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:56.393440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:56.394340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:15:56.395186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:56.398142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 217/435] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:56.405877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.416476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2e3b2640-82f1-49e7-9747-6ec977355def None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:56.418401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2e3b2640-82f1-49e7-9747-6ec977355def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:56.475850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.510115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 219/436] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:56.517973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56deae-85f0-42ae-b521-c40449f16ce3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.518298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56deae-85f0-42ae-b521-c40449f16ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.518570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56deae-85f0-42ae-b521-c40449f16ce3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.520791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 220/437] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:15:56.525909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.526188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.526455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.553220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207584c-3407-47ca-b01b-71864c9c729a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:56.553718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:15:56.555447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:15:56.558718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.595390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.596122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.596427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.596691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.597346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:56.597642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:56.599524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:56.609081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 218/438] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:56.615134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.645938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 221/439] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:15:56.726958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f49d2e-0722-4f51-a600-e57013de8e90 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:56.745974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.763380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.763983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.764350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.764626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.807342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.833567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:56.835297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:56.908336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b9b1-4aea-494f-a3cb-8353ab9e531b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:56.942307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:56.942934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:56.943212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:56.943481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:56.950350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:56.955349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:56.958699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 219/440] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:15:56.959541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:56.960629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:56.961522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:15:56.962373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 222/441] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:15:56.965128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:56.980073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.059925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1332dd-8c09-4acc-86d8-47068e5c761c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.066533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d75cb5-8788-4f7f-bb92-0cfe394c7eeb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.100836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.101034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.101452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.101560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.101726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.101851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.101995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.102089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.103105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.104569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.106168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 223/442] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:57.109180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.113098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:15:57.117929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:15:57.118857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:15:57.119924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 220/443] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:15:57.120624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.176329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.198030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.221504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c49b-da9a-4670-af65-d07c6c4cf6a8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.223119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04e68-bf5b-4bc3-8b13-95c61082cbff None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.257730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.258348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.258509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.258603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.258779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.259153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.259427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.259515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:57.259693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.259785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.260813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.261103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.262224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.263764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 221/444] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:57.264532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.274870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 224/445] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:57.274870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.366213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323b0e-f0ac-4535-9213-e0734120717e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.366590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a28e7-aaf6-41c2-8454-10d2e4747a81 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.393884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.401663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.401829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.402261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.402378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.402536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.402634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.402805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.402901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.403518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=1e1b8d5397754a2e93980b3202ff7e7f, scope.project.id=267a872030254d2c8a0328ffe37ea732) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:57.403625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:57.403820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.403922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.405242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.405341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.411007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 225/446] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=1e1b8d5397754a2e93980b3202ff7e7f => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:57.411775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.418062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 222/447] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:15:57.419128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.431569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.533255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc8ba-1ea1-4de6-9210-4221b90e9ff5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.541354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ebca-dc4e-430b-a345-782405783399 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.572667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.573290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.573567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.573844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.583022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.583636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.583910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.584179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.653749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.658857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.660226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.667089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.669100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.670432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.775318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 223/448] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:15:57.780843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.790491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 226/449] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:15:57.796048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.882220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2ddd6-3757-4352-98c0-1b0d7d9bc59c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.883453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.892994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f012ad6-4729-4783-8b69-52abce20db04 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:57.893482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:57.918416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.919070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.919350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.919625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] PATH_INFO: `/v3/projects/3af173ac0b7d40069de1fa09770699a3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.920776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] RBAC: Authorizing `identity:get_project(project_id=3af173ac0b7d40069de1fa09770699a3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.922208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.923852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 224/450] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/projects/3af173ac0b7d40069de1fa09770699a3 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:57.927012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:57.927625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:57.927894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:57.928274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] PATH_INFO: `/v3/projects/267a872030254d2c8a0328ffe37ea732` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:57.929290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:57.929393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] RBAC: Authorizing `identity:get_project(project_id=267a872030254d2c8a0328ffe37ea732)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:57.930771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:57.933296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 227/451] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/projects/267a872030254d2c8a0328ffe37ea732 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:57.937468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:58.029095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6597-61a0-4068-b5fb-41689f3a9e5f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:58.038075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e17c-c180-49af-8630-ffe6559dd15c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:58.062078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:58.062630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:58.062905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:58.063710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] PATH_INFO: `/v3/groups/3ad883a183f145ce9f8560850fd88e48` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:58.064895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] RBAC: Authorizing `identity:get_group(group_id=3ad883a183f145ce9f8560850fd88e48)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:58.066338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:58.067877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 225/452] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/groups/3ad883a183f145ce9f8560850fd88e48 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:58.070935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:58.071548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:58.071819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:58.072089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] PATH_INFO: `/v3/groups/d947b3a27d2242e78e6588ff42521051` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:58.073444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] RBAC: Authorizing `identity:get_group(group_id=d947b3a27d2242e78e6588ff42521051)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:58.073628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:58.074843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:58.081822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 228/453] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 05:15:57 2022] GET /identity/v3/groups/d947b3a27d2242e78e6588ff42521051 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:58.083309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:15:58.111358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:58.124988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:15:58.179305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1346d-e3a7-498d-b0e3-da90ca388f8a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:58.181627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5796-9b10-4591-b10e-6ce3af610b34 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:15:58.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:58.214236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:58.214513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:58.214778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:58.215543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:15:58.215669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] RBAC: Adding query filter params (group.id=3ad883a183f145ce9f8560850fd88e48, role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=3af173ac0b7d40069de1fa09770699a3) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:58.215854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:58.216138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:15:58.216452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:15:58.216724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:15:58.217308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045ac3e-a20d-41e0-b361-28441061f862 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:58.217441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] RBAC: Adding query filter params (group.id=d947b3a27d2242e78e6588ff42521051, role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=267a872030254d2c8a0328ffe37ea732) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:15:58.217747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:15:58.219160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf80ae-54ef-466c-a0b3-64c9f68a8daa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:15:58.222931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 226/454] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:58 2022] GET /identity/v3/role_assignments?group.id=3ad883a183f145ce9f8560850fd88e48&scope.project.id=3af173ac0b7d40069de1fa09770699a3&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:15:58.231061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 229/455] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 05:15:58 2022] GET /identity/v3/role_assignments?group.id=d947b3a27d2242e78e6588ff42521051&scope.project.id=267a872030254d2c8a0328ffe37ea732&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:00.979002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bc9c-f28c-46ba-9893-65b1366edc61 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:00.979558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bc9c-f28c-46ba-9893-65b1366edc61 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:00.979558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bc9c-f28c-46ba-9893-65b1366edc61 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:00.981172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 227/456] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:00.986447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e09a9-a9d7-4ef8-9ef0-f41e9746f129 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:00.986904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e09a9-a9d7-4ef8-9ef0-f41e9746f129 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:00.986991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e09a9-a9d7-4ef8-9ef0-f41e9746f129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.013927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-774e09a9-a9d7-4ef8-9ef0-f41e9746f129 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:01.015099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-774e09a9-a9d7-4ef8-9ef0-f41e9746f129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:01.110589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 230/457] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:01.265788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26100b-4234-48bd-b6c4-f77d6d664ca5 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.266924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26100b-4234-48bd-b6c4-f77d6d664ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.267342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26100b-4234-48bd-b6c4-f77d6d664ca5 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.268839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 228/458] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:01.274074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b800-f021-43eb-ade8-97f7634b13de None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.274357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b800-f021-43eb-ade8-97f7634b13de None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.274625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b800-f021-43eb-ade8-97f7634b13de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.300682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7875b800-f021-43eb-ade8-97f7634b13de None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:01.301866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7875b800-f021-43eb-ade8-97f7634b13de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:01.328040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:01.397398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 231/459] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:01.539904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26100b-4234-48bd-b6c4-f77d6d664ca5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:01.654821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26100b-4234-48bd-b6c4-f77d6d664ca5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:01.660283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a3681-ddfc-4be2-861d-e76c94686fce None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.660283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a3681-ddfc-4be2-861d-e76c94686fce None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.660283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a3681-ddfc-4be2-861d-e76c94686fce None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.661822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 232/460] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:01.665443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:01.693698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cc3f6-a58b-4d60-8e98-57516c6d3aa1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.694306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cc3f6-a58b-4d60-8e98-57516c6d3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.694767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cc3f6-a58b-4d60-8e98-57516c6d3aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.696142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:01.696801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.697078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.697351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.716588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:01.718057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:01.731758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-752cc3f6-a58b-4d60-8e98-57516c6d3aa1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:01.732967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-752cc3f6-a58b-4d60-8e98-57516c6d3aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:01.744411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:01.745595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:01.746443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:01.746443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:01.746443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:01.750629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 229/461] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:01.752133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:01.816192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 233/462] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:01.817207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f51ad7-8863-4561-beec-2e86e3c80d3e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.817488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f51ad7-8863-4561-beec-2e86e3c80d3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.817756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f51ad7-8863-4561-beec-2e86e3c80d3e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.819081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 234/463] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:01.821310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f51ad7-8863-4561-beec-2e86e3c80d3e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:01.881449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2efd5d-a954-42b2-935e-9fdbc90c50ec None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:01.918043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:01.919035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.919516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.920029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.920965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:01.921486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:01.924374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da363e-aef0-434a-9959-47930d793bf5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:01.942592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:01.942977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 230/464] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:01.944606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:01.945092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:01.945562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:01.959434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f51ad7-8863-4561-beec-2e86e3c80d3e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:01.968484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:01.970771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:01.987442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.010218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.010841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.011273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.011678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.025839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.028269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.056275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 231/465] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:02.056275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:02.056523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:02.057347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:02.058192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:02.058192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.058192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:02.058192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 235/466] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:02.061208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.193711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc365a-f6d5-4b61-8811-faaacad57696 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.218006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377f30d-93fc-495d-bc31-0030dda1c1f9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.229595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.230403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.230840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.231217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.231978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.233537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.259590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.260266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.260552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.260873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.261515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:02.261675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 232/467] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:02 2022] POST /identity/v3/users => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:02.261901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.262677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.263431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.265353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.280899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.304163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 236/468] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:02.308831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.407628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c6f89-20ac-4133-9ed2-f1503a4c375a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.460569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba31864-9de0-4625-a2f6-16df4c911b9d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.471563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.473152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.473641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.474104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.490625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.493000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.504350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.504920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.505199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.505472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.506140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.507586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:02.510693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:02.511581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:02.512517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:02.512517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.512517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:02.513630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 233/469] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:02.514300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.538993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 237/470] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:02 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:16:02.539735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.597209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.612203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.649694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6a02c-5d8d-4786-b225-4d1dd9b1a633 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.690348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181482f-fc79-4f7d-89ef-361365104bd4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.694793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.695745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.696162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.696597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.710987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.712770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.730706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.731318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.731595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.731864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.732537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:02.732838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fc78bfad-fa50-45e2-a8a7-8c9a6c04e2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:02.733471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:02.734353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:02.735949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:02.735949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:02.735949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:02.736137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972c6d-94e9-4d4d-b563-9def0fe756f2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.737877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 234/471] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:02.739431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77406e-355c-48ad-9e09-4c620786d7c3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.739956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77406e-355c-48ad-9e09-4c620786d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.744763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77406e-355c-48ad-9e09-4c620786d7c3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.746361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 235/472] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:02.746981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77406e-355c-48ad-9e09-4c620786d7c3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.749442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 238/473] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:02.751835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.752117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.752423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.771713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:02.773959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:02.868295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 239/474] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:02.873255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:02.876523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77406e-355c-48ad-9e09-4c620786d7c3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:02.887562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.914938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:02.915609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:02.915890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:02.916216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:02.916896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:02.917198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:02.918729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:02.923410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:02.936139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 236/475] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:02.945195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:03.013052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfc465-d5c2-4bb0-94fa-50e888b03550 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:03.049714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:03.050479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:03.050821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:03.051161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:03.051874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:03.053519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:03.092282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 240/476] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:02 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:03.100674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1adb-3e78-44be-a3de-d1a3ea779b88 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:03.157760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:03.158441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:03.158724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:03.158995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:03.159665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:03.161219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:03.189304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 237/477] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:02 2022] POST /identity/v3/users => generated 249 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:03.216425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:03.350294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:03.357505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5fcd4-369e-4448-948d-b2c63b4cada5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:03.396645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:03.397272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:03.397548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:03.397815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:03.398523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:03.399965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:03.408729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.sql.core [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:16:03.408729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 05:16:03.408729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [parameters: {'id': '0b2ca948f4984c63b6c2c50a0f223fe0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 05:16:03.408729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 26 05:16:03.426934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return ref.to_dict() Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:16:03.427874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application yield resource Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise self.value Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application session.commit() Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:16:03.428807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:16:03.429655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:16:03.430541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:16:03.431350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [parameters: {'id': '0b2ca948f4984c63b6c2c50a0f223fe0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:03.432206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 05:16:03.435149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:03.437184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 241/478] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 05:16:03 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:03.437319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:03.470191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:03.552421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed3a89-f151-428d-8223-2cc7b6885fc0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:03.589304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:03.589958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:03.590234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:03.590504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:03.602886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-85064d77-40cc-45c9-85d5-2e0ef9283044 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:03.609054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:03.610092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:03.610940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 238/479] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:03.617827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:03.775085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4d2ab-e4d7-4b3b-b239-e118018e9a62 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:03.814760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:03.815730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:03.816210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:03.816731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:03.817583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:03.818106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:03.819683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c81ed7-ac55-472f-8bfd-15eac5721147 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:03.829113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 242/480] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:05.212777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ddbae-68d3-40e5-8b7d-21b09b73ec5d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.213455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ddbae-68d3-40e5-8b7d-21b09b73ec5d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.213455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ddbae-68d3-40e5-8b7d-21b09b73ec5d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.216279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 239/481] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:05.221903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.222399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.222573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.240110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb009-310c-417b-961d-2a41ef427ac3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.240437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb009-310c-417b-961d-2a41ef427ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.240688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb009-310c-417b-961d-2a41ef427ac3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.242001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 240/482] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:05.248281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.248281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.248281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.252691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:05.257162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:05.277801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:05.279200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:05.307005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:05.390587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 243/483] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:16:05.411943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 241/484] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:05.725421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:05.804642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:05.827772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:05.855878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54981897-35e8-4a92-a237-12be13eb3f35 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:05.892815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:05.893676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.894120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.894527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.905433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9d7e769b-a7eb-41cf-9957-e8ed2c46affc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:05.910751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:05.911778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:05.912600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:05.914763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 244/485] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:05.916232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67ea8-345c-46f3-a645-5ad97f21b527 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.916665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67ea8-345c-46f3-a645-5ad97f21b527 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.917070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a67ea8-345c-46f3-a645-5ad97f21b527 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:05.918412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 245/486] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:05.924086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a67ea8-345c-46f3-a645-5ad97f21b527 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:05.934331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf29c-2ba3-4095-a995-0211c1a83b27 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:05.941911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:05.986768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:05.987310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:05.988525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:05.989173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.003959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-8711b603-f8d0-4571-99e1-066ad3c7aaa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:06.010438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:06.011327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:06.012167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:06.014026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 242/487] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:06.015381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.015661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.016512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.034796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:06.035815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:06.055159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a67ea8-345c-46f3-a645-5ad97f21b527 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.099100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.099717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.099995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.100292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.100964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:06.101270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.102656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56675f-bd2c-4a2e-b969-3f7645ec807e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.108804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 243/488] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:06.110158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.118898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 246/489] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:06.118898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91646e-992b-48c6-8f41-32c47d7e9f1d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.118898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91646e-992b-48c6-8f41-32c47d7e9f1d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.118898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91646e-992b-48c6-8f41-32c47d7e9f1d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.118898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 247/490] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:06.121326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91646e-992b-48c6-8f41-32c47d7e9f1d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.139685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.211504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.241341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91646e-992b-48c6-8f41-32c47d7e9f1d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.246280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fa338-29ca-46cf-a029-de100564f80a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.287644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.288489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.288762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.288959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.289327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.289825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.290178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:06.290407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.290663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.290959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.292484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507422cd-e8f6-4ad2-af05-294e3a1652bc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.303868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.305564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.306290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 244/491] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:06.308128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.308592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.308992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:06.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:06.323469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:06.324336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:06.324336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:06.324336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:06.326147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 248/492] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:06.332055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.333295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:06.334556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:06.405704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.420528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 245/493] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:06.421290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.441862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b51461-b55b-41ae-b8e6-2719073514b3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.470680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.480333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.481172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.481625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.482037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.494093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.495998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:06.511437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:06.512590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:06.513532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:06.513532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:06.513532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:06.515173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 249/494] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:06.516920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.537108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f3afc-78c6-4212-8d40-60f68d463a13 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.575661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.576331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.576902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.577020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.577616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:06.577933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.580074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.599139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 246/495] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:06.600571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.629457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d97ab-3b27-4f56-b843-4ea6c21c5f01 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.663873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.664812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.665296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.665747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.665869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.677749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:06.683463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:06.684621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:06.685594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:06.687392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 250/496] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:06.689227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.713059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba2c7-d205-4be6-aff0-9cd9a51f1591 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.729328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.747972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.748531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.748811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.749074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.749700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:06.749995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.751910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.763234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 247/497] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:06.764017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.792986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8f8d5-36e1-424d-9a46-4655e6e8856c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.826007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.826616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.826889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.827154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.834419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.835864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:06.848224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:06.849153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 251/498] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:06.850151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.864056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73592e58-9525-4d2d-9545-da0dbbc608ca None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.882042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.898525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.899126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.899400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.899671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.900350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:06.900652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:06.902017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09265cd7-f2cd-447e-9d2d-18336ecb3e75 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:06.909737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 248/499] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:06.910859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90153c-c724-4efa-ab5a-4553c5bf836f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.911140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90153c-c724-4efa-ab5a-4553c5bf836f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.911414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90153c-c724-4efa-ab5a-4553c5bf836f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:06.916421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 249/500] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:06.916421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90153c-c724-4efa-ab5a-4553c5bf836f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:06.950699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:06.955151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01d80d-e112-4c2a-b79a-83a63a1547a2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:06.988776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:06.989390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:06.989664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:06.989954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.001162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:07.006383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:07.007452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 252/501] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:07.008394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.020755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90153c-c724-4efa-ab5a-4553c5bf836f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.063735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.064518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.064796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.065065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.072499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.073918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:07.087212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:07.092895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 250/502] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:07.094127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.115597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcced-20bf-43e1-8760-0ce07ee4b9a9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.134311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.147018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.147628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.147902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.148174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.148927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:07.149229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.150676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3c0c-c028-4fc2-8658-4640c87be3ee None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.186790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 253/503] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:07.188717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.189049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.189324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.191270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26f7f1-e277-4601-b659-da8a4100a07a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.206672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:07.207860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:07.218366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.224681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.225300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.225574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.225864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.236457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.237907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:07.252413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:07.253810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:07.254668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:07.254668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.254668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:07.254668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 251/504] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:07.254668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.280022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 254/505] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:07.280961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.351077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.361372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be3714-317f-4e45-b463-4d13381587ca None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.386500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64169f2-181b-4fa7-9a11-d626ead97589 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.401531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.402402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.402680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.403063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.403835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:07.404135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.405540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.415522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 252/506] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:07.416362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.424906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.425510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.425780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.426101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.426827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:07.427145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.429003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.446610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 255/507] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:07.447635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.453115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.528009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3891-84b0-4f0e-8a4c-65df3844b722 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.559506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50ca8-0b00-4467-9f4e-c7e603b79d83 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.563129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.563775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.564051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.564352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.587812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.593133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.593733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.594624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.594925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.595558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:07.595855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.597259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.608271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 256/508] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:07.609060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.630535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.632053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.677939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.714910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45351-c08a-456a-ade5-6ac14ade5cda None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.737232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 253/509] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:07.738101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.755057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.755690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.755967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.756241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.766779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.768219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:07.782575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:07.783568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:07.784473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:07.784473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.784473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:07.784473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 257/510] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:07.784473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:07.796585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.841398 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436408f-1d24-4d16-8996-c72800fbcb22 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.874660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.875792 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.876056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.876363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.881653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31a636-788e-4252-a08e-42e34ccc392d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:07.897572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:07.916220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:07.916880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:07.917158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:07.917432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:07.927739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.929210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.942870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:07.943556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:07.944506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:07.945346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:07.945346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:07.945346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:07.945519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:07.945587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 258/511] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:07.945893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.000870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.045215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 254/512] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:08.045396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764c0d-63f3-488c-b9df-07b436703b34 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.045969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.080171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.080873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.081154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.081420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.089718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.091151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:08.103660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:08.104626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 259/513] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:08.106082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.134421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.152517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ff1c-8d11-4347-9c18-565713f0ff7c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.185296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.185918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.186188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.186454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.196830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.198255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.206160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407028d8-3e19-4508-a2e1-ef7ee4aa482c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:08.212377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:08.213273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:08.214406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:08.214406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:08.214406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:08.214406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 255/514] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:08.214406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.223972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.240765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.241373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.241650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.241920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.242547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:08.242855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.244222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.256512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 260/515] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:08.257295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.318464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423abb1-c9ca-444b-83a0-da0fecaf8935 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.345677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.350786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.351387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.351666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.351940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.352602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:08.352901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.354257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.358479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e661-7c83-4f32-ac21-a1774f113184 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.366475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 256/516] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:08.367183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.397019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.397644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.397920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.398189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.408907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.410370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:08.425303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:08.426365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:08.427776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:08.427776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:08.427776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:08.427776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 261/517] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:08.427776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.440376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.475420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a99e5-3a0a-4249-807a-09d13b189a6b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.510569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.511185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.511519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.511798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.512558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:08.512863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.514281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.530766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b90fc-4940-4e5b-9a88-a19e906dd6db None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.550423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 257/518] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:08.551240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.568637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.569177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.569462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.569724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.570382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:08.570682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.572051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.581274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.589201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 262/519] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:08.589201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.653970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633e531-72b4-4a1d-b717-d2c1bb77c2a6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.677284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.683212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1ffb4-1ef2-4f74-bd0c-812d43d5a5b1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.687478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.688100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.688409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.688675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.695464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.696940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:08.709394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:08.710488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 258/520] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:08.711400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.718825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.719953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.720301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.720582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.721227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.722648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.743177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 263/521] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:08 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:08.744087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.816988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.830334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9a6a-ca96-4291-98af-813bcaf0833d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.868533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5d65-e943-413d-b38f-a4a183113196 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:08.869329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.870737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.871233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.872701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.873536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:08.874004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.875582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:08.893712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 259/522] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:08.897310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:08.907713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:08.908718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:08.908998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:08.909286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:08.952270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:08.984053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:08.986168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.006928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee153-5723-48c7-9aaa-577fe8674a35 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.041033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.042642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.043333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.043671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.044056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.051556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.053015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:09.064875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:09.065931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 260/523] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:09.066767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.101893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 264/524] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:09.101893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.179291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4831cf-c95d-4329-a47a-73953aed2e91 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.181721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.202735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1f2f-07bf-445d-aecf-16de3d84fd8e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.218815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.219974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.220230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.220517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.221241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.221637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.223006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.239080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.239709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.239982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.240271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.244011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 261/525] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.245264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.247321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.249049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:09.261724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:09.262791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 265/526] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:09.263640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.285764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.361979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2db33-ab98-433a-abd3-8a2992601b91 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.370272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e9100-39fd-40b6-b5bf-82f1b9477aea None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.399186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.399989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.400453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.400960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.401778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.402224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.403914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.404147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.404567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.404846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.405118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.415989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.417458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.425278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 262/527] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.426384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:09.437913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:09.439079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:09.440053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:09.440053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:09.440053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:09.440053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 266/528] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:09.440372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.454210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.515842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.538318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61456673-86aa-4516-a137-c58f9671ceab None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.551633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca6fde-67c0-4b02-9ad5-0a70b1ae8aeb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.574203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.574803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.575073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.575338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.586847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.587945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.588233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.588677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.589364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.589675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.591112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.608854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 267/529] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.610053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.650891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.652357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.680655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.716019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c91c8-317d-41a0-ba93-961aba6f2f1f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.744760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.754805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.755416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.755696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.756032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.756792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=0da5e4ab96bb4a9584682f32cbafb2e7) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.757102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.758534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.763804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 268/530] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/role_assignments?user.id=0da5e4ab96bb4a9584682f32cbafb2e7&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.764597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.766671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 263/531] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:09.767723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.898830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bb751-6078-410f-8f6d-b1c72572f4c5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.902777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafab87-a652-4be0-a2fa-87fdde3f9ebb None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:09.937121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:09.937322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.937937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.938225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.938497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.939366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.939481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.940916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.959824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 269/532] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.959824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:09.960849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:09.961699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:09.962113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:09.962535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:09.963403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:09.963851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:09.965498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:09.980875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 264/533] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:09.980875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:10.094602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:10.183030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ee8e-cb6c-4ada-b57b-41e36ab96664 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:10.197977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb50d7-22fa-4264-bd3c-f99191a9d03c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:10.243481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:10.244655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:10.245092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:10.247204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:10.259192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:10.261227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:10.265632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:10.266495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:10.266926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:10.267331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:10.288503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:10.290179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:10.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:10.292880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:10.293754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:10.302676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 265/534] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:10.308118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:10.315985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:10.316940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:10.317774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:10.317774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:10.317774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:10.319816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 270/535] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:10.327422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:10.416324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:10.508382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42e50-badd-4ea3-86da-d89a36a28d2b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:10.593241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:10.594614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:10.594814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:10.595069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:10.595768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:10.596068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:10.597533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:10.606645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8bd21-6aa4-4c79-b70e-968a32fd071c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:10.614520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:10.653253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 266/536] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:10.663467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:10.714236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:10.715780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:10.719374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:10.719374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:10.719374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:10.719374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:10.719374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:10.782300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 271/537] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:10.789688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:10.889740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3dc45-3041-4687-9dc3-20cf2ad260c2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:10.900075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:10.934388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:10.934999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:10.935305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:10.935538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:10.936284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=a938ac6d7d3b42ab9e8aa8b2b9ea9410) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:10.937474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:10.940163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:10.952411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 267/538] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:10 2022] GET /identity/v3/role_assignments?user.id=a938ac6d7d3b42ab9e8aa8b2b9ea9410&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:11.104479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff0a75-4854-43f9-bb91-2e9c841d2345 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:11.162834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:11.163841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.164348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.164833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.178407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:11.180170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-08f9206f-61ec-4b40-bfa3-05b708ee4eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:11.196424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:11.197498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:11.198519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:11.200510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 272/539] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:11.207883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:11.261914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e428a8-2df8-4103-a02f-31bf6e64b294 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.262452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e428a8-2df8-4103-a02f-31bf6e64b294 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.262864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e428a8-2df8-4103-a02f-31bf6e64b294 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.264678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 273/540] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:11.270200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.270662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.271087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.311716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:11.313645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:11.352623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:11.360638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064cdd-3809-4910-b006-d33a7db6670d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:11.398399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:11.399005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.399291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.399565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.400271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:11.400652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:11.402508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:11.415902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 268/541] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:11.425552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:11.449867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 274/542] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:16:11.494968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:11.592041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf0ba-95d2-40a9-b055-ca2ee8757e81 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:11.638576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:11.639195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.639488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.639762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.640518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=0102a128d81741c88deeeafedbf0d390) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:11.640861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:11.642291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:11.648157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 269/543] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:11 2022] GET /identity/v3/role_assignments?user.id=0102a128d81741c88deeeafedbf0d390&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:11.723784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:11.871563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281be8-a1bb-4272-9bc5-70b6c576d0f5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:11.910645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:11.911201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:11.911478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:11.911746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:11.922848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:11.927771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:11.928674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:11.929456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 275/544] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:11.935950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.066137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:12.074315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d0f1-c7bb-4f9f-9efb-6dcd12340edf None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:12.124899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:12.125784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.126068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.126347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.127024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:12.127331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:12.128768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96701c24-f698-4499-8137-f76ae61f2ce5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:12.137808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 270/545] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:12.143938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.173845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:12.292402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ff65-e8f3-46e1-9b8b-299a0550fcb0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:12.325143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928fd4-374a-4aea-a0f1-121d89dcbfc6 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.325470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928fd4-374a-4aea-a0f1-121d89dcbfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.325708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928fd4-374a-4aea-a0f1-121d89dcbfc6 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.328469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 271/546] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:12.332876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.333166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.333448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.347684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:12.348626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.349875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.350366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.367671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:12.369758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:12.372485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:12.373701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:12.384989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:12.385893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:12.386724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:12.386724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:12.386724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:12.388798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 276/547] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:12.394387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.497247 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 272/548] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:16:12.514939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:12.568321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c8a2-09f7-488d-86e0-2d0f39cf9e5e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:12.622568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:12.623588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.624060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.624582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.625948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:12.626441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:12.628065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9cc4-296f-48ee-b2d1-f29b8fee5979 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:12.644875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 277/549] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:12.650635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.713695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183596-9f2c-432f-a880-49db4fde14a3 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.714313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183596-9f2c-432f-a880-49db4fde14a3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.714777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19183596-9f2c-432f-a880-49db4fde14a3 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.716528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 278/550] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:12.721740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.722280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.722739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.760789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:12.763243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:12.765445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:12.773440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89c9af-7eea-4c82-803b-2d07b9357d69 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:12.817831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:12.818482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:12.818767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:12.819038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:12.829103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:12.831400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:12.847451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:12.848632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:12.849542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 273/551] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:12.850123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.854712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 279/552] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:12.861951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:12.963585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3901fe8-10be-4ce7-8973-38f6f195900d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:12.972469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.003263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.003890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.004171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.004471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.015629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:13.021643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:13.022708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:13.023541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 274/553] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:13.029279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.045484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.065293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40b1ad-f5b9-4791-a1bd-eb77e84bb909 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.133225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.134170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.134640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.135125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.136614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:13.137829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.140135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.162343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 280/554] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:13.171474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.225017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124689d-8bbd-46ba-9b4b-bdebcde56b05 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.279519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.280520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.280994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.282122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.283166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:13.283662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.285731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.297507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 275/555] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:13.298702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.343399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7cdd-70f9-4975-a8c8-981146ebd4aa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.363099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.384302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.389779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.391205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.391713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.392178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.428640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8c049-d67d-4d75-8866-cf5bd76ca138 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.467875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.469931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.471541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.472415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.472797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.473276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.484967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:13.492050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:13.493053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:13.493869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 276/556] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:13.494866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.587940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 281/557] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:13.589159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.605875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.619093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1569a1-17aa-4e79-8c3b-4f0ebc7f0b94 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.649153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.659492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.660365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.660863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.661010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.672989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.674595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:13.692611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:13.693629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:13.694509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:13.694509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:13.694509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:13.699207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 277/558] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:13.700234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.714609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709030-ff23-4268-92de-fd760a79caed None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.754738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.755455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.755739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.756066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.756880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:13.757186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.758699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.769441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 282/559] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:13.770452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.815932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb517585-a6a3-415f-a3ae-1c38efb01ad0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.851025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.854285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.854914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.855193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.855467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.867896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.869631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.876235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d89d9-afb5-4070-beae-d463ac73ac60 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:13.885462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:13.886393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:13.887314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:13.887314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:13.887314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:13.887314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 278/560] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:13.889052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:13.916071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:13.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.917056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.917251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.917928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:13.918191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:13.919537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69aa4-5250-4fab-8c63-d40ef925c899 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:13.923619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 283/561] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fe72-61af-4c5d-adad-01bc99e520a2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fe72-61af-4c5d-adad-01bc99e520a2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fe72-61af-4c5d-adad-01bc99e520a2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 284/562] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:13.936711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fe72-61af-4c5d-adad-01bc99e520a2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.010484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aba0e-4c39-4d8c-ae6f-30e6b560a298 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.046459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.047622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.047898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.048160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.049816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fe72-61af-4c5d-adad-01bc99e520a2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.059596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.061194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:14.079213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.081402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:14.082384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:14.082384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:14.082384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:14.082384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 279/563] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.085952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.091405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.092034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.092348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.092626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.093301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:14.093602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.095705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00ad2e-9349-4014-a567-9abeb14236cb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.099464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.114594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 285/564] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:14.114594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.114594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.114594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.131618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:14.132820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:14.191159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.201997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 286/565] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:14.202829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.209647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b81a3-a226-46d3-bc0c-c2f906ed18d4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.245916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.246560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.246834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.247106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] PATH_INFO: `/v3/users/glance` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.254910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.256819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:14.269421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.270563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:14.271640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 280/566] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.276210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.310109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d9f5-70b1-46c9-b397-f3cc39987406 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.338051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.349822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.350695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.350974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.351394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.352803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:14.353116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.354502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4473-f1e7-4582-9ceb-f06a95b2faeb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.366373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 287/567] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:14.369763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27239-4308-48cc-a428-4f637c772ae5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.370048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27239-4308-48cc-a428-4f637c772ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.370322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27239-4308-48cc-a428-4f637c772ae5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.376834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 288/568] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:14.376834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27239-4308-48cc-a428-4f637c772ae5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.387987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcab8d8-3f3d-4103-acc6-c966fd436bef None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.424521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.425146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.425403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.425667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.432997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.434887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:14.447895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.448952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:14.449841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 281/569] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.450317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.455247 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.489007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27239-4308-48cc-a428-4f637c772ae5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.523053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.523655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.523991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.524286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.524996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:14.525299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.526720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aabb53-ea0c-4d77-8f9a-3d52772cfc06 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.543776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 289/570] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:14.544972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.545248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.545512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.554400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb6e09-222a-458d-896d-a59fcb52f58e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.564607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:14.566092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:14.588978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.589935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.590215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.590485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.597852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.599275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.605099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:14.610365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.611302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 282/571] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.612820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.635278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 290/572] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:14.636037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.673662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.712415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdc6f-d16d-49a6-a22d-3b336b8f9869 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.742959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282a0db-6958-406f-90e3-2072c3551a15 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.746020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.746620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.746897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.747168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.747852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:14.748155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.749600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.762201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 283/573] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:14.763004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.785160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.785850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.786132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.786405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.797729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:14.802829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.803786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 291/574] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.805000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.840167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.872422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536fee3-2e6a-4f63-9e2a-37b126eaf9e0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.898430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:14.909662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8aea-5f40-4ffa-98ea-9828a9bc34c9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:14.910907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.911697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.912576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.912857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.922831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.924312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:14.939297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:14.940277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:14.941193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:14.941193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:14.941193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:14.941193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 284/575] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:14.941193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:14.946544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:14.947278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:14.947556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:14.947824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:14.948610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:14.948920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:14.950344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:14.967856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 292/576] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:14.969296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.043589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d654025-bc3a-49a3-80bc-65f593603827 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.057582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.067187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954a90c-6dd9-4c94-b485-803c79765d5e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.077405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.078544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.081643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.081643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.081643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=ed5f73bac406439e856d745885c0c092) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.081643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.081643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.087106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 285/577] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/role_assignments?user.id=ed5f73bac406439e856d745885c0c092&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.087992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.103747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.107305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.107305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.107305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.107305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.107305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.108870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.119946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 293/578] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.120776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.153138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.227081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e626913-4d31-4699-ab13-b89e36c90980 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.237130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3b3eb-768b-4726-8239-7133fdc7233d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.266772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.267500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.267774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.268042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.275389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.275989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.276289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.276562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.277196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.277492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.278847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.295130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 294/579] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.295311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:15.295407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:15.296484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:15.297823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 286/580] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:15.298888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.319919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.409120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c229d01-c683-4267-a996-9a6ec58a7b2e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.415199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.418340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2823b9-bb43-46d2-870b-b03303182090 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.444464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.445106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.445371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.445642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.458233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.458422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.459042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.459406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.459681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.459864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.470512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.471935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:15.478993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:15.479934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:15.480855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:15.480855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:15.480855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:15.480855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 295/581] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:15.481091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:15.492148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:15.493186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:15.494768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:15.494768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:15.494768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:15.494768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 287/582] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:15.500434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.548967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.588591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312adc-70de-47b5-93d7-c6ac7262420d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.607702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc4fec-a3b2-4d19-9566-f045c34730d2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.623677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.624333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.624612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.625069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.631914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.633386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.643621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.644219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.644518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.644786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.645437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:15.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:15.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:15.647934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:15.648403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.648474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 296/583] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:15.648474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.652539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.665656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 288/584] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.665656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.680570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.756979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fbc21-a419-4d54-91be-bd0510647126 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.771066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ca29a-d47f-4981-a6d7-54834007b25a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.785710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.795738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.796466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.796819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.797249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.797736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.798051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.799447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.809420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.810039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.810326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.810597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.811246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.811550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.816658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 297/585] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.816658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.816911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.833924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 289/586] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.834808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.911874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7ba5-20b7-4e32-b8c4-71dad78f9bdd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.915333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:15.942192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4986d7-9c6d-4a3a-8c58-8680c255f6ff None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:15.947566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.948222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.948532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.948803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.949633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:15.949935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.951501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:15.969337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 298/587] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:15.969337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:15.982715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:15.983364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:15.983647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:15.983922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:15.995979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:15.997512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:16.012977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:16.013936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:16.015127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:16.015127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:16.015127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:16.015127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 290/588] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:16.016110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.024952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.086319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4bec-c449-4c2f-99bc-50f8d5435776 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.122698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.123321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.123601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.123896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.126192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cb4f2-4271-4d77-9957-6c4747e6f3ec None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.131685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.133357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:16.146665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:16.147761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:16.148773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 299/589] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:16.149255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.159276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.169515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.170136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.170411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.170678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.179395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.181092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:16.193829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:16.194798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 291/590] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:16.195751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.261096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563d2b-0066-406d-ac7f-b5db4063213e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.268658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.300143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.300918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.301266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.301609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/0da5e4ab96bb4a9584682f32cbafb2e7/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.307452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406354a1-9b72-4204-9fec-6ea1e142695f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.320412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=0da5e4ab96bb4a9584682f32cbafb2e7, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.322419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.342085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.342382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 300/591] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:16 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/0da5e4ab96bb4a9584682f32cbafb2e7/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:16.342688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.342966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.343179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.343282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.343904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:16.344206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.345732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.364465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 292/592] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:16.365898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.420242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.481251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a6833-d803-4a3c-8e88-2cec5489e209 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.499523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3e941-0704-42cf-979e-5c969b4ee685 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.521833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.522618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.522978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.523345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.524182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:16.524612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.526341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.551631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.552275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.552556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.552826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.553351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 293/593] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:16.553516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:16.553813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.554235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.555250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.572004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 301/594] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:16.573006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.596979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.674922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa29c8e-9171-4070-883f-4df82763e1a1 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.678585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.718435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a9e8-a0a2-4dca-b2fc-96f413594f5b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.721033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.721679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.721959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.722230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.732191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.734471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:16.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:16.748483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:16.749489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:16.756997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 294/595] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:16.757986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.763567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.764211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.764520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.764795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.779514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.781323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:16.801903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:16.803331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:16.804228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:16.804228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:16.804228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:16.805649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 302/596] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:16.806276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.878424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ee72c-2807-4900-8e50-98b966ca6b4c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.897953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.920607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.921512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.921841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.922122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.940921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057efde-b4b4-446b-8da1-6d422ab0f195 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:16.943029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=2eadef8d5ac94a3eb3da8d0d3a66d7b5, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.945458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:16.962327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 295/597] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:16 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:16.963360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:16.982991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:16.991342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:16.991975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:16.992279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:16.992563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:16.993387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:16.993712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:16.996199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.024650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 303/598] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:17.027706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.171180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a3096-95a7-42fb-aa28-f6f9334e4292 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.202512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a0a20-a360-43fa-9cf7-b2210c1b17a9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.221009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:17.232453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.233137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.233435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.233720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.245972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.247439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.251495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.252163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.252473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.252751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.253415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:17.253705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.255070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:17.264210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:17.265274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:17.266723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:17.266723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:17.266723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:17.266723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 304/599] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:17.273100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.278704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 296/600] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:17.285411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.388513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:17.407335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f157-7d0d-4487-a3f3-93bef97fac1e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.435302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4cfb3-1c46-491e-b368-3e42f5fbb86d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.448225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.448864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.449142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.449440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.450162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:17.450468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.451834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33aa0ed-bb50-4d87-8348-697e9c70371d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.469995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 305/601] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:17.469995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf328-8c30-4eb8-8859-8881f498cb3a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.469995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf328-8c30-4eb8-8859-8881f498cb3a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.469995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bf328-8c30-4eb8-8859-8881f498cb3a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.469995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 306/602] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:17.473275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bf328-8c30-4eb8-8859-8881f498cb3a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.489022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.489577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.489856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.490180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.499052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.500922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-6aeb8de8-9679-4dca-b280-027467e60d7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:17.514960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:17.516448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:17.517405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 297/603] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:17.517930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.517930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.523196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.541883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:17.542219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:17.544391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:17.632276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 298/604] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:17.632276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.632548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bf328-8c30-4eb8-8859-8881f498cb3a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.709028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.709679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.709956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.710226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.721423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.723213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.728271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:17.744224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:17.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:17.746347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 307/605] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:17.754941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.795187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bf6ef-2b50-42da-83be-3a3035f35a4a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.862717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.863395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.863669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.863939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.864790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:17.865096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:17.866530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:17.891772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 299/606] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:17.898746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:17.941612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:17.947795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99187-c59b-4de1-af85-2d97156dbcfd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:17.996324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:17.997316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:17.997809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:17.998285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:17.999189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:17.999707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.001741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.019824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 308/607] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:18.021048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:18.055019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92688807-55c2-4fcb-9654-4c8a51534f15 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.093387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:18.112069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.113054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.113496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.114025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/a938ac6d7d3b42ab9e8aa8b2b9ea9410/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.131942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=a938ac6d7d3b42ab9e8aa8b2b9ea9410, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.133741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.150429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1680-4534-4699-abb7-ce1c977e064f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.151023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 300/608] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:17 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/a938ac6d7d3b42ab9e8aa8b2b9ea9410/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:18.152069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:18.218634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.219813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.220195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.220596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.245145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:18.251210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:18.252147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:18.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:18.256622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 309/609] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:18.261277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:18.282435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704328-4cd0-4ad1-a1fa-91223da315d9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.297652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:18.326982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.328889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.329601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.330095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/0102a128d81741c88deeeafedbf0d390/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.347016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=0102a128d81741c88deeeafedbf0d390, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.349003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.364444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 301/610] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:18 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/0102a128d81741c88deeeafedbf0d390/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:18.436720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce87e-9f6a-43f4-9790-132f5fb47190 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.492774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.493395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.493668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.493941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.494860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:18.495159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.496567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.506226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 310/611] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:18.512061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:18.549890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:18.653065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e5a9-e413-417e-b428-eb819241d156 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.694020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.694660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.694944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.695219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.706857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.708586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-01d7d827-b4d2-485a-9603-678953d4800c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:18.725346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:18.726368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:18.727278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:18.727278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:18.727278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:18.727278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 302/612] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:18.733287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:18.756394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337bba2-e3a7-49f2-b815-d12332972dee None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.756900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337bba2-e3a7-49f2-b815-d12332972dee None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.756900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337bba2-e3a7-49f2-b815-d12332972dee None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.758589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 303/613] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:18.766183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.766445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.766716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.802804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:18.803999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:18.876289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:18.911795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7784d-c848-401f-982b-ac5b0b32d3f7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:18.934340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 304/614] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:18.967222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:18.968945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:18.969439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:18.969873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:18.970696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:18.971156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:18.972697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:18.989707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 311/615] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:18.995305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.004452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:19.105601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a777-1032-4ca7-874c-fd71a6c0440b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.162176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.162810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.163094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.163364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.170777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:19.172862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:19.184735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:19.185744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:19.186709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 305/616] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:19.193420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.289091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.345183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48acf2-2b51-46e9-a608-b6dc6cde78c4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.386470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:19.419031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.419644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.419920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.420188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.420954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:19.421259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:19.423302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:19.432866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:19.445015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 312/617] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:19.451130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.457885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cc0e8-7682-4f9e-83bb-b7a20397b0a6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.526057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.527127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.527615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.528439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.550627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:19.562727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:19.563755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:19.564661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:19.569437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 306/618] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:19.575235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.639566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456cad4-622f-4f2a-839f-ab9f7b16f0ab None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.679217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.679850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.680128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.680448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.691724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:19.693194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-93e5ac0f-d6d1-44cf-8d45-431ab89ca839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:19.708802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:19.709947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:19.710841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:19.710841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:19.710841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:19.710841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 313/619] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:19.715864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb50-8b5d-4fa8-a981-0e1add118c3d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.715864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb50-8b5d-4fa8-a981-0e1add118c3d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.715864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb50-8b5d-4fa8-a981-0e1add118c3d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.715864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 314/620] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:19.716874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bcb50-8b5d-4fa8-a981-0e1add118c3d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.728640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cdc4-86bf-4323-8f67-81a2da9fb036 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.760550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:19.770910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.771521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.771788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.772052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.773166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:19.773467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:19.775166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b63a94-fca6-4082-a408-78965511ec9d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:19.785067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 307/621] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:19.785304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:19.786893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.787176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.787682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.812575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:19.815293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:19.854178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bcb50-8b5d-4fa8-a981-0e1add118c3d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:19.900197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:19.900869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:19.901194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:19.901473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:19.902112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:19.902417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:19.903788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:19.904396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 308/622] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:19.905744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:19.922912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 315/623] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:19.930527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.068853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8966d-c0f6-4aa0-8df7-382c175b6c1e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.088681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.096034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e976d2-a221-43e3-afc0-c3bcac24c65b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.129587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.145834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.146476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.146840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.147119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.153150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.154173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.154653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.155122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.158516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.160359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.170046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.172442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:20.177036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:20.177985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:20.178892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:20.178892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:20.178892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:20.178892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 309/624] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:20.183732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:20.188648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:20.189557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:20.190459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:20.192794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 316/625] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:20.196788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.307062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07153-7221-4c9f-905a-330d26d7be0c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.317255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62362e-cd76-4f9b-8772-d827ecf7455e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.346070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.346684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.346963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.347234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.347870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:20.348187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.350353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.354624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.355486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.356003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.356528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.364130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 310/626] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:20.369012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.375601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:20.382255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:20.383204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:20.384040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 317/627] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:20.385393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.408483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.427419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.503091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a108e8-a5e1-4718-b9d5-fe2a99cc51fe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.507396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f032c-03ed-4e28-b3e9-37ac9445de8b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.543914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.544847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.545290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.545704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.548998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.549762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.550136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.550517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.551304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:20.551710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.553318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.555656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.557713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.568005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 311/628] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:20.568984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:20.575639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:20.576735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:16:20.577811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:20.581094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 318/629] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:20.582071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.685111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.690523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.701842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe2a9b-1860-4225-8cdc-6c9b98448281 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.702258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff8d07-a97e-48d7-9c0d-7e79fddbb788 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.740095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.740883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.741159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.741469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.742156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:20.742459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.744075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.752078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.753470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.753948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.754427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/ed5f73bac406439e856d745885c0c092/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.759412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 312/630] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:20.759412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.784875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=ed5f73bac406439e856d745885c0c092, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.787260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.805320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 319/631] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:20 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/ed5f73bac406439e856d745885c0c092/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:20.806605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.882117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e44ba-9494-49bb-a927-4140a503233f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.938081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.939113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.939597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.940408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.942020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:20.942846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:20.944542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afc50d-4fa4-4bad-a107-e74f06784072 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:20.944915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f70421-fd9a-4a8e-9669-6c351afbf552 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:20.969312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 313/632] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:20.971148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65faeea-0bdc-43d8-83f6-1d4622c99e45 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.971646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65faeea-0bdc-43d8-83f6-1d4622c99e45 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.972110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65faeea-0bdc-43d8-83f6-1d4622c99e45 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.974367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 314/633] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:20.976080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65faeea-0bdc-43d8-83f6-1d4622c99e45 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:20.986507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:20.987159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:20.988129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:20.988129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:20.992473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:20.998525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.001311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.003477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8c038df3-f615-4f6e-a3a4-411027f0d9ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:21.023528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:21.024777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:21.025684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:21.025684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:21.025684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:21.026027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 320/634] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:21.030330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.031594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.032121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.053952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:21.055502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:21.119064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65faeea-0bdc-43d8-83f6-1d4622c99e45 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.135545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 321/635] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:21.140828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.163804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.164765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.165199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.165629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.242755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.244984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b988b-825f-4354-9c02-3e6464d956b0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.258114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.260760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105e03d-67cb-439f-8560-11381a280f84 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.285665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.298802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.299526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.299872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.300208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.300990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:21.301362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.303047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd932c6-1e3a-42f6-beb1-7a4dee497128 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.315925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 322/636] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:21.317542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738442ee-79ab-4401-bdf4-197ae02d7c58 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.317904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738442ee-79ab-4401-bdf4-197ae02d7c58 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.318249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738442ee-79ab-4401-bdf4-197ae02d7c58 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.323423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 323/637] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:21.323423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738442ee-79ab-4401-bdf4-197ae02d7c58 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.363458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 315/638] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:21.365052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.365505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.365926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.385854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:21.387403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:21.436371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738442ee-79ab-4401-bdf4-197ae02d7c58 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.469037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 316/639] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:21.470694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.473518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.474523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.474523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.474815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.482609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.484805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:21.497083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:21.498112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 324/640] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:21.499045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.525057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.551859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.599755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b66bf1-dc6d-4bf0-9e60-8f1c68670aa5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.613764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563779-4cc0-4d90-a72f-4c1581d6cfc8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.651199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.651976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.652375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.652646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.654879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.655608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.656045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.656460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.665528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.667244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.667645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:21.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:21.675007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:21.675845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:21.676512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 325/641] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:21.677765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:21.692147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:21.693749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:21.695589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:21.695589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:21.695589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:21.695589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 317/642] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:21.695589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.795877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd752c-c873-4c06-ad98-2e6cb8dd86a1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.798175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.803788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:21.816099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2815ad-96c3-47bc-b2c4-c68d35257e13 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.831786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.832425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.832698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.832962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.833674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:21.833968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.835404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.852125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 326/643] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:21.852932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.858361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.859764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.860430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.860883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.862156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:21.862653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.864915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:21.876472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 318/644] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:21.877582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:21.958305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8487f8-be69-4968-98a3-50b4db494de5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.989931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c79-246f-479d-a011-cec2f5b06e5e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:21.993958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:21.994817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:21.995090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:21.995359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:21.995987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:21.996313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:21.998073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.014828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 327/645] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:22.016458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.032341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.032971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.033406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.033661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.040461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.044140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:22.050294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.056200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 319/646] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.057451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.070995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.137056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b5ca-39c3-448c-b6b5-d32ae64e39aa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.172058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.172689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.172967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.173242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.173363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21465-463d-4d89-928a-7b60b046653a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.210464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.211089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.211365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.211632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.222720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.224140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:22.241336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.242305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:22.243198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:22.243198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.243198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:22.243198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 320/647] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.243948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.251482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.252991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.275609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.316787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.358465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 328/648] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:22.361781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.363403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c305691-7bd4-4fe3-a2c1-0068dfe10a14 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.406887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.407522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.407805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.408094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.417935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.419355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:22.432966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.434066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 321/649] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.435218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.476043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10938a04-27bb-4fe3-a8b0-ff8f3b843daa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.497986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.513378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.513975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.514250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.514514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.515679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:22.515969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.517381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.525778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 329/650] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:22.529119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.541667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761833-b0ef-4efb-80fc-f82b80452812 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.569010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.578426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.579031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.579312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.579579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.580210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:22.580543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.581892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.593808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 322/651] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:22.595660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.642702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e986031-86fb-4898-8f76-9f4c83136ecb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.678932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.679566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.679844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.680114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.693965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.695908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.706822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fe3-f54a-4446-9449-b37bccc661ce None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:22.715393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.716415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:22.717321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:22.717321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.717321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:22.717321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 330/652] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.718035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.745625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.746230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.746504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.746771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.747457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:22.747753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.749292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.760600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.760957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 323/653] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:22.761939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.810570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:22.837983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55209cf9-3e23-462b-8041-29d74845e678 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.866942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d569af-c4ae-4c9f-8132-6c543d67f846 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:22.872609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.873232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.873562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.873927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.885405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.886952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:22.902293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.903503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:22.904927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:22.904927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:22.904927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:22.904927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 331/654] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.904927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:22.906706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:22.907334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:22.907603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:22.907867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:22.922224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:22.924030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:22.935822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:22.936851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 324/655] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:22.938782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.007143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.024747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fb6a-94d0-4c93-8cf9-563c92df1e40 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.052098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144417f0-1fe9-4b85-996b-37a4cc62b442 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.059725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.060601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.061002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.061225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.061575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.062875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.063258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.064694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.081785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 332/656] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.081785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.098819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.099614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.099894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.100169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.100936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=0da5e4ab96bb4a9584682f32cbafb2e7) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.101250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.102661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.109471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 325/657] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:22 2022] GET /identity/v3/role_assignments?user.id=0da5e4ab96bb4a9584682f32cbafb2e7&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.110486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.196490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4de27-4f3e-4122-8ed0-f4969389bc37 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.229372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891803e-01fb-4d44-adbc-51e437fefdc8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.234451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.235315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.235687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.235950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.236651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.236953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.238481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.251474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 333/658] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.252411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.263340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.272626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.273500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.273923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.274445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.275471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.275921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.277589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f2d2e-6136-41b7-813e-634cfee65596 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.299320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 326/659] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.300643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8c3a-455c-4f20-bca3-91f5ac5c7f8d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.300934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8c3a-455c-4f20-bca3-91f5ac5c7f8d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.301209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8c3a-455c-4f20-bca3-91f5ac5c7f8d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.302550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 327/660] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:23.303077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8c3a-455c-4f20-bca3-91f5ac5c7f8d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.335182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.374016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230093-b98a-4491-bc0a-a41177d31f0f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.409887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.410516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.410952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.411090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.417964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.419446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.420580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8c3a-455c-4f20-bca3-91f5ac5c7f8d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:23.431757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:23.432721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:23.433601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 334/661] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:23.434073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.464710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.465476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.466100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.466442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.474261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.475706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4a603ac2-fd23-4f4c-9529-678eeda630da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:23.488273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:23.489236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:16:23.490268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:23.494009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 328/662] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:23.495332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.495616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.495889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.509959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.514183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:23.515700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:23.553463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd0198-7c1b-4fc4-972b-9ddcfb7741ec None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.596290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 329/663] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:23.596290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.598316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.599001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.599397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.599734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.615224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.693235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.695400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.703499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c086-0e41-4a4b-9d5b-d70d006fae22 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.741659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.742703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.743125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.743533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.744890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.745352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.747750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.773967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 330/664] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.775013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.779059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.827354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 335/665] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:23.829039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.886926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:23.911677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4ef-99da-4796-969f-f5fc2a744467 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.943995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e50bfa-ea3a-4871-a9ac-bc2c6c9cc7a6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:23.946101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.947321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.947321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.947321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.947931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.948241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.949681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.969309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 331/666] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.970495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:23.985412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:23.986023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:23.986298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:23.986565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:23.987284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:23.987592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:23.989056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:23.995631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 336/667] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:23.996230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.039819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.102663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb05f-8f64-4fad-8d56-bcab144bbb36 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.144008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d2a1-65ea-46eb-a37c-0891379867af None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.155074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.155761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.156632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.156909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.160829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.171770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.173982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.183186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.184444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.184444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.184672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:24.193585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:24.194812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:24.195946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:24.195946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.195946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:24.202693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 332/668] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:24.202693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.206423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:24.211844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:24.212815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:24.213645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:24.216115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 337/669] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:24.217160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.330380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.335659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643fdc-863f-4751-890c-a1168e101cb5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.345765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600eaaa9-41d8-44e5-b1f9-ac1983f1cc4e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.375227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.376199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.376714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.377174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.384766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.385440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.385787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.386152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.386853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:24.387220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.388752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.402042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 338/670] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:24.407011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.454419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.464473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.466713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.525830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a14977-81fa-4f05-993e-4d25b08804b6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.562877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.563982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.564512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.564988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.565876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:24.566515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.568280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.577669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 339/671] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:24.579448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.595595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.600683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 333/672] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:24.602144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.696117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa51d5b-6c2b-45f1-93ee-dd9303f217f7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.711864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.728262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9dcbd-a232-4094-967d-c1f885d47919 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.733227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.734364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.734990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.735461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.742863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.744826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:24.756887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:24.757813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:24.758710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:24.760831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 340/673] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:24.761807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.769593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.770267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.770545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.770808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.784239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.786073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:24.804740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:24.805738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:24.806631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:24.806631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.806631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:24.811032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 334/674] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:24.811997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.846327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:24.891647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fd7e3-a5f5-46cd-a4c1-5d8e53b2a67b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.940075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddc0a8-7e40-487c-a656-6d4bf803b50e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:24.944574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.945501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.945970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.946434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.962286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.964447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.976365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:24.977231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:24.977478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:24.977902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:24.978757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:24.979105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:24.980664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:24.985469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:24.986454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:24.987364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:24.987364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:24.987364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:24.994135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 341/675] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:24.994135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:24.996628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.001374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 335/676] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.002191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.118656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dffb1-97ff-4c9d-9246-d8723c2ea338 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.125979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aed84-8a26-4bc9-9f5c-1ff301eb9be9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.148423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.163798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.164794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.165315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.165827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.166780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:25.167239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.169128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.169579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.170545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.171014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.171446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.172183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:25.172881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.174073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.184762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 342/677] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.190766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.191279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 336/678] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.192203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.263849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.321673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210951c-7b6c-46e6-92ed-2bec6d2c4d68 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.325298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2862-0f52-4ebf-a6f6-94a0dd0dfb26 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.357655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.358264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.358540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.359334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.360366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=a938ac6d7d3b42ab9e8aa8b2b9ea9410) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:25.360678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.362170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.368496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 343/679] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/role_assignments?user.id=a938ac6d7d3b42ab9e8aa8b2b9ea9410&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.369619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.370248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.371117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.371587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.372165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.381385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.383358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-743e7454-08bb-49ad-8008-730aa7249873 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:25.398152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:25.399184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:25.400604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 337/680] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:25.405265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba47-f8da-46d8-b901-c8c07efe8cf1 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.406417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba47-f8da-46d8-b901-c8c07efe8cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.406758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bba47-f8da-46d8-b901-c8c07efe8cf1 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.408282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 338/681] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:25.413355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bba47-f8da-46d8-b901-c8c07efe8cf1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.448296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.509702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae484-f75b-4394-b146-69a4b69d58aa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.543592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bba47-f8da-46d8-b901-c8c07efe8cf1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.549577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.550650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.551129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.551691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.560015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.561810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.568405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e9c2be2c-c927-46ef-9929-9edd674069ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:25.577337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:25.578308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:25.579205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 344/682] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:25.580297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.580297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.580540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.599499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.600492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.600976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.601405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.602124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:25.602231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:25.602430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.603292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:25.603848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.625383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 339/683] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.626235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.699741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 345/684] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:25.699741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.778399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.782993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8dc33-83ac-4ff3-8f34-162ded7e02f9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.820738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:25.821294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:25.821576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:25.821953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:25.822650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:25.822956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:25.824491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:25.850101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 340/685] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:25.852810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:25.914793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd1757-7381-43d5-b856-5704081ac421 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:25.924620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:25.989084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d8458-7fc0-445a-8fc5-339d74ef043c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.016952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.017948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.018434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.019003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.024782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.025386 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.025660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.025927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.026644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=0102a128d81741c88deeeafedbf0d390) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:26.026946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.028377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.035253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 341/686] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/role_assignments?user.id=0102a128d81741c88deeeafedbf0d390&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:26.036113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.099086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.101013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.148223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:26.240061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe52648-4500-4f06-ba1a-00ab80f9f1a7 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.242499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:26.286419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 346/687] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:26.291863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.314656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.315813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.316671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.317147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.336326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.338958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:26.363262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:26.364229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:26.365813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:26.365813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:26.365813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:26.367958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 342/688] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:26.375421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.447919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1153d-3715-47a3-aedc-97bf40ac73a4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.488060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.488747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.489025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.489307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.497368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:26.502791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fd75-899f-41a8-a5d1-94e3fba90d1d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.504962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.506413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-50d7f119-879d-4045-ab97-a6318ebde876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:26.531751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:26.533000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:26.534124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:26.534124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:26.534124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:26.534124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 347/689] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:26.535229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99aab3-dd79-4a3b-9c8a-9614e42b8760 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.535547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99aab3-dd79-4a3b-9c8a-9614e42b8760 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.536052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99aab3-dd79-4a3b-9c8a-9614e42b8760 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.538262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 348/690] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:26.540998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99aab3-dd79-4a3b-9c8a-9614e42b8760 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.548855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.549476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.549756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.550029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.550691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:26.550980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.552445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d339f1-8038-41ba-8a7e-fa8ca119a2f9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.566470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 343/691] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:26.567704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.567990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.568274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.590389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:26.591596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:26.603282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:26.676907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 344/692] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:26.677730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.689182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99aab3-dd79-4a3b-9c8a-9614e42b8760 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.755620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.756244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.756542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.756814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.757546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:26.757851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.759225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:26.779136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 349/693] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:26.784937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:26.873192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f75d3c-c17b-400c-87b6-31dbba529158 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.887279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:26.947574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96454d-1a14-4840-a677-50b0747e92da None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:26.962095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:26.962708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:26.962989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:26.963258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:26.963954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:26.965413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.001844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:27.011490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 345/694] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:16:26 2022] POST /identity/v3/users => generated 254 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:27.012774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.014491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.015486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.016036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.016768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.032312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.034275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:27.066959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:27.067939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:27.068936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:27.074183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 350/695] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:27.078581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.219629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbce8b-f57e-4f12-914d-bb869d7002ad None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.256715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.257605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.257723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.258028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.269053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:27.275300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:27.276224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:27.277134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:27.280381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 346/696] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:27.280501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9ed85-eec9-4002-abd3-26a0f5b86d8e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.283220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.294490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:27.323349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.323963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.324241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.324594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.325402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:27.325711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.327272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.342255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 351/697] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:27.348378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:27.349717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.396497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbf984-400d-4368-b52e-bcf708796cd4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.442444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.443057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.443337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.443608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.444312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:27.444622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.445989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.456557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 347/698] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:27.461954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.485949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fadb11-2ffb-4d36-8a70-561384eba426 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.525883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.526614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.526878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.527176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.527874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=ed5f73bac406439e856d745885c0c092) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:27.528183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.529848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.544527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 352/699] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/role_assignments?user.id=ed5f73bac406439e856d745885c0c092&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:27.580677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5fb36-7ced-46a8-be4d-9bd185d65430 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.606279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:27.621151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.621853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.622133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.622391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.634993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.636475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fb3c9904-6c07-4dd8-9f28-a02ac952780b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:27.653905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:27.654980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:27.656105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:27.656105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:27.656105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:27.656105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 348/700] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:27.661676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:27.721745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:27.783097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378359-3ef7-4271-9421-e10d2784be34 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.783295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378359-3ef7-4271-9421-e10d2784be34 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.783541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69378359-3ef7-4271-9421-e10d2784be34 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.785196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 349/701] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:27.790772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.791061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.791320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.820804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:27.822374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:27.845419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b9450-7afd-41d6-9a19-5e7b5f57cc85 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:27.903059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:27.903676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:27.903954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:27.904370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:27.905033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:27.905339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:27.907805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:27.913211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 350/702] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:16:27.926344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 353/703] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:27.931568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.047352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bb568-c4ae-428d-a312-54614d828bc0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.049734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.089105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.089816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.090098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.090372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.098217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.099760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:28.113276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:28.114354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:28.115318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:28.116760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 351/704] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:28.122433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.205489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.238872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.247652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0d627-e2ca-400d-8761-7c546333e0da None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.321955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.322490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.322766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.323037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.323768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:28.324070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.325538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.353758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9f4d8-ad3c-4b4f-82aa-f8038297f73b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.360497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 354/705] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:28.365710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.392823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.394895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.395174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.395444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] PATH_INFO: `/v3/services/network` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.396045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.397825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:28.421058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:28.422691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:28.423529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 352/706] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:28.429206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.475669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.548804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2858de0-8511-47c7-8a44-3ed6e20acd4e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.598320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f83570-737c-42a7-80e0-b083fb4038ae None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.613182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.614163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.614632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.615090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.648612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.676957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.677572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.677844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.678109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.678802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] RBAC: Adding query filter params (name=network) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:28.679028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.680419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.692692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 353/707] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:28.698019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.700168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.702240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.817515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 355/708] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:16:28.823361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:28.836702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.854541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160d901-84c3-4db7-9c47-faa4d038ac4c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.887199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:28.913241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:28.913664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.913945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.914218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.914843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:28.915157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:28.916525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e8c57-887a-4c05-bcc2-16e23710f1b6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:28.926062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 354/709] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:28.965371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228cc59-b1ad-49dc-8e80-b94d8175de86 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.966034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228cc59-b1ad-49dc-8e80-b94d8175de86 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.966612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228cc59-b1ad-49dc-8e80-b94d8175de86 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:28.967226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a71a5c-db55-4d26-b365-3ea036fa076c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:28.970005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 355/710] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:28.975367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f047451-9d98-4986-9832-9c486468670c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:28.975753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f047451-9d98-4986-9832-9c486468670c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:28.976128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f047451-9d98-4986-9832-9c486468670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.002693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8f047451-9d98-4986-9832-9c486468670c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:29.009061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8f047451-9d98-4986-9832-9c486468670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:29.028493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.029149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.029421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.029698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.052234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.053735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:29.090854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:29.091880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:29.092810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:29.092810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:29.092810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:29.092810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 356/711] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:29.101568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.161831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 356/712] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:29.258945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b9ea4-8c82-47e0-ac44-b0b3ff2d0785 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.261480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:29.295598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.296789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.297067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.297405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.298063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:29.298364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.299752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.313528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 357/713] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:29.320358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f047451-9d98-4986-9832-9c486468670c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.417258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:29.442118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f047451-9d98-4986-9832-9c486468670c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.486340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.486990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.487267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.487626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.499343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.501693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.502844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:29.516949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:29.517876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:29.518811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 357/714] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:29.525372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.690587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc35e4e-5623-4fa9-a1b2-1736b5f67634 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.695577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903b4-c1cd-4c38-b69b-5d23a60eaaea None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.743336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.744409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.744694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.745300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.745898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.747278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:29.759343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:29.760388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:29.761238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 358/715] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:29.764642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.765290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.765740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.766578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.767207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.767906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:29.768210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.772475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.798199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 358/716] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:29.807111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.819432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:29.832630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:29.932587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298357b3-28fd-4fcf-9e14-1e1d9c014a68 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.934248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f873cc2-5342-4fb4-8f63-bb2bbf7576b3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:29.970019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.970629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.970907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.971180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.971851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:29.972216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.973630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec6671-f076-42d9-aa25-be418b170e67 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.976136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:29.977246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.977723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.978166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.978729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 359/717] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:29.979271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:29.979807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:29.980275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeff76a-a5f4-4d94-8294-514add58a3e2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.980798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeff76a-a5f4-4d94-8294-514add58a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.981146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeff76a-a5f4-4d94-8294-514add58a3e2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:29.982000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2c410-cff6-4718-a9f9-52bf5705b44c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:29.987075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 360/718] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:29.988205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeff76a-a5f4-4d94-8294-514add58a3e2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:29.989216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 359/719] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:29.990720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:29.991183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:29.991618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.011530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:30.013133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:30.098124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:30.100059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 360/720] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:30.230051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:30.242345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeff76a-a5f4-4d94-8294-514add58a3e2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:30.279578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:30.280194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:30.280748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:30.280940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.281741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:30.282046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:30.283418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:30.289461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 361/721] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:30.465301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:30.594400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f7f3f-269b-4340-816f-b8819ae8fe6e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:30.633362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:30.633995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:30.634268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:30.634535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.646044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fe183153-91ed-467d-8b65-6af3d76d674a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:30.651090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:30.652236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:30.653301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 361/722] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:30.660810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:30.664945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02418bcd-b5d4-4cd0-8f69-9beaa2fa733d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:30.665273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02418bcd-b5d4-4cd0-8f69-9beaa2fa733d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:30.665543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02418bcd-b5d4-4cd0-8f69-9beaa2fa733d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.667736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 362/723] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:30.676825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:30.676825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:30.676825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.701276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:30.702499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:30.779775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a039e1f-e3e0-4f38-9f10-0162a625db38 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:30.802333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:30.818109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:30.819105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:30.819592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:30.820127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:30.821233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:30.821727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:30.823311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:30.830592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 363/724] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:30.833427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 362/725] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:30.841284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:30.993649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4e19d-0934-4b98-ace2-7d3660973273 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.035369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.036010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.036313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.036590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.048648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.050117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:31.071217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:31.072244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:31.073710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:31.073710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:31.073710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:31.076720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 364/726] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:31.087605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.128705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:31.234995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d06319-b57c-4a97-91a5-b8d2578188cb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.270015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.270541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.270814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.271080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.271713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:31.272010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.273400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.286634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 363/727] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:31.292006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.325607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.354417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:31.416622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:31.444650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370558-7e96-4b8d-9c42-1c7ea0b6d674 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.445204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e541-519e-4347-bf86-a78a9e6ec6f8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.481306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.481986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.482265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.482536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.493951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:31.499959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:31.500926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:31.501773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 364/728] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:31.512742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.519806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.520585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.520752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.521003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.533123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.534599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:31.564677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:31.566896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:31.569183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 365/729] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:31.578148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.635135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635aa49-6820-4e6e-b6a1-c843936b9dc3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.678118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.678727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.679005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.679279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.679935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:31.680361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.681883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a6d5-824b-454c-84ea-319403773465 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.687884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:31.692490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 365/730] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:31.693723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac06c3-ee23-40d8-88f4-8c2ee0984984 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.694378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac06c3-ee23-40d8-88f4-8c2ee0984984 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.694847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac06c3-ee23-40d8-88f4-8c2ee0984984 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.700494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 366/731] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:31.701478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac06c3-ee23-40d8-88f4-8c2ee0984984 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.739266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367104-929e-4ae5-98ba-63a3a10885ca None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.741132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:31.783307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.783975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.784788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.784788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.785546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:31.785924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.787553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c167990-0a0b-41a4-8a48-a87b8cffc58d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.806150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 366/732] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:31.807606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.807962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.808496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.826071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac06c3-ee23-40d8-88f4-8c2ee0984984 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:31.828668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:31.829923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:31.869945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:31.870893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:31.871344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:31.871783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:31.885447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:31.887092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:31.902719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 367/733] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:31.907169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:31.908112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:31.909029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:31.909029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:31.909029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:31.909615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:31.910963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 367/734] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:31.917735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.018941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.061758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.086971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6470-a8b2-4536-af42-6be79186907b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.108581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd5356-fb51-4893-93ee-8d3815be6da6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.152157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.152788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.153064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.153338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.170641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.171302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.171581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.174134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.174943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:32.175260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.176763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.206494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 368/735] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:32.217201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.244958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.246387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.353923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.365919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 368/736] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:32.366491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3a62-0410-4e67-8df2-ea49b33a7806 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.366798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.382598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.407200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.408362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.408836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.409320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.418314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.420383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:32.433991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:32.435047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:32.435983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:32.439760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 369/737] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:32.440978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.497655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eda5dd-f325-4f33-8fba-8de654d0e88d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.536698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.538685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.539241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.540052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.541239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.543088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.563648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 369/738] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:16:32 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:32.564824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.565539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa4a09-70e7-4a14-89c6-540cd5779c0b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.610892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.611471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.611745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.612011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.623933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.625692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.634122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0f0d8ac6-19c2-425a-9a67-9f83df472b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:32.643360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:32.644770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:32.645665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:32.645665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:32.645665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:32.645665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 370/739] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:32.646364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125fc40-b50d-4a67-867f-d0280aad5c00 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.646558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125fc40-b50d-4a67-867f-d0280aad5c00 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.646828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125fc40-b50d-4a67-867f-d0280aad5c00 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.648963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 371/740] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:32.653252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125fc40-b50d-4a67-867f-d0280aad5c00 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.682208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.720878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7baa-f1dd-4a70-93fc-c898a5968ad8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.761569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.762193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.762470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.762744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.763481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:32.763782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.765507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8954fd-c0a3-47fa-a09e-58a1c0601480 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.787498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125fc40-b50d-4a67-867f-d0280aad5c00 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:32.790084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 370/741] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:32.791422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.791771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.792050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.815384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:32.816579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:32.830636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:32.831345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:32.831596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:32.831872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:32.833805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:32.833805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:32.834292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:32.851731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 372/742] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:32.853001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.904034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 371/743] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:32 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:32.904991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:32.981778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:32.982184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.005235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b4d9-1f05-4966-96b4-bf62b62a13ab None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.048188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.048999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.049277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.049545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.058412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12534a-e2cf-47df-9cde-ea3b8235d858 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.102752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.103609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.103732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.103935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.117129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.118705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.129441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.131051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1bf49-052f-4746-b3b2-ac4e3da55711 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-3763fb02-79e0-4d4a-8d5f-f905bbc403f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:33.131222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:33.132274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:33.133388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:33.135037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 372/744] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:33.136373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3827cf-c1db-4eec-98d6-08caa90d3d89 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.136751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3827cf-c1db-4eec-98d6-08caa90d3d89 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.137846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3827cf-c1db-4eec-98d6-08caa90d3d89 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.142845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 373/745] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:33.142845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3827cf-c1db-4eec-98d6-08caa90d3d89 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.225464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.247242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 373/746] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:32 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:16:33.248594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.248883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.249190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.255576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.264238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3827cf-c1db-4eec-98d6-08caa90d3d89 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.267182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:33.268396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:33.301165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.301989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.302405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.302835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.303652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:33.304118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.305554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.318248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 374/747] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:33.319070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.343833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 374/748] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:33 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:16:33.345012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.447638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8af5-c5d4-45f0-82b4-379614c39aec None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.448840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca00e7e-0c74-40aa-b20e-4f7a214e7ead None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.461125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.487091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.487944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.488442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.488930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.490686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.491490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.491775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.492047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.501525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.503250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.505035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:33.511208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:33.512209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:33.513118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:33.516384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 375/749] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:33.517791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:33.525007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:33.525976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:33.526927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:33.526927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:33.526927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:33.528727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 375/750] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:33.529756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.543509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.646240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0bde-6cbf-4ffc-b5c1-34eb8a3a1323 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.652996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326dad0-2c07-4753-bca3-512f22083538 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.679573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.680159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.680489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.680761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.681479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=85e1f5c17df9495182082a9d85fa3401) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:33.681786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.683222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.688664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.689050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 376/751] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/role_assignments?user.id=85e1f5c17df9495182082a9d85fa3401&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:33.689545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.689985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.690364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.691115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:33.691518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.693035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.697235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.705190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 376/752] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:33.706320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.738930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.831519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:33.845065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a0c3-5f71-4368-9739-55c63fe9f5ad None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.857503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76016ec-6f1d-4640-bfc6-0fc761eba359 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:33.888124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.888787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.889183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.889459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.891303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.892953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.902597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:33.903570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:33.904090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:33.904634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:33.906206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:33.906767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:33.908510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:33.912802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 377/753] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:16:33 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:33.920421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:33.933779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 377/754] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:33.940418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.083229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:34.085487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd8ad4-4bb0-4cac-a18b-763a877b8383 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.109096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4a8f-baa5-43c1-afda-9d4b0f8b2971 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.132768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.133810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.134404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.134899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.143483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.149904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.149904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.149904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.149904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.151903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.161150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.162626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:34.174597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:34.175559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:34.176509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:34.179243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:34.180282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:34.181749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:34.181749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:34.181749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:34.181749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 378/756] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:34.183084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 378/756] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:34.184866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.186417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.190023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:34.308359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c72ad5-de5a-48d3-b6e6-1eb8e2cd1732 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.331060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f470d27-dccf-45ec-a675-21a3044f194d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.341510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:34.351176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.351176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.351176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.351176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.352161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:34.352161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.353602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.372050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 379/757] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:34.377754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.396682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.396682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.396682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.396682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.398893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:34.398893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.398893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.428347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 379/758] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:34.437830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.529781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5522-d6d9-4eed-aac0-f53acf3f5511 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.582349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.583271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.583962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.584444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.585390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=ed5f73bac406439e856d745885c0c092) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:34.586501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:34.587555 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.589296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.598034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 380/759] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/role_assignments?user.id=ed5f73bac406439e856d745885c0c092&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:34.662018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6d1d8-36eb-4bce-8f42-4cf663ee51ed None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.704081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.704770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.705050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.705506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.713906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.715431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:34.729631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:34.731074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:34.732056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 380/760] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:34.742197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:34.826271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:34.888435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df346cb-bc7c-4575-8085-4ad072f98fd4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:34.939919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:34.940987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:34.941480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:34.941945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:34.943002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:34.943505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:34.945411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952ee1-198d-4693-aaa4-d73d42cc82a0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:34.972847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 381/761] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:34.979064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.127355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c8aa7-5273-4111-9a03-fe3301864ad5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.127691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c8aa7-5273-4111-9a03-fe3301864ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.127946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c8aa7-5273-4111-9a03-fe3301864ad5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.129605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 382/762] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:35.146884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.147162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.147353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.175781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4319-a6f2-4318-a2c2-dbf000e93f7c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:35.182805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:35.184454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:35.229047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:35.253889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:35.254542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.254821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.255091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.285225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:35.286681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:35.311582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:35.312868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:35.313970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:35.313970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:35.313970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:35.315770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 381/763] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:35.316840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 383/764] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:35.329224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.470534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265709f-5e7b-4672-9295-00ea1a15d0d0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:35.518885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:35.519545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.519821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.520139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.520806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:35.521106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:35.522467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:35.538651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 382/765] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:35.543992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.617477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:35.687717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066beac-5ec1-4902-beba-db4784a6249e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:35.691699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.728874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:35.733874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:35.734726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.735006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.735274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.742534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:35.743991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:35.764129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:35.765387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:35.766362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:35.773739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 384/766] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:35.782343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.833008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf8f3-69fb-4d2e-a63d-b0fb337c5850 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:35.897899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:35.898528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:35.898798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:35.899061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] PATH_INFO: `/v3/services/network` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:35.899642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:35.901015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:35.916941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:35.918000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:35.918836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 383/767] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:35 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:35.925808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:35.978772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec2a2-d6a8-4e32-94f6-a7ea7ac0cdd8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:35.982447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:36.043889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.044805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.045226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.045641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.046532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:36.047006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.049969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.072304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 385/768] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:36.073256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:36.124617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:36.127884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679f2c-3fe8-431a-8128-4780ee612825 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:36.190263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.190924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.191200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.191466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.192147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:36.192471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.194096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.203219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 384/769] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:36.208677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:36.271432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642c3a1-f157-47b5-8d0c-cde50f53c391 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:36.337766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.338429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.338702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.338972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.342150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=2eadef8d5ac94a3eb3da8d0d3a66d7b5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.343641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efae78c-4b9a-4ec0-ae61-d83c9f855d92 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.365904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 386/770] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:36 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:16:36.386764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26d450-54cb-4269-bfc4-15b559f457c5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:36.428262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.429039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.429322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.429593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.430225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] RBAC: Adding query filter params (type=network) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:36.430525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.431880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.435183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05bc8c-e0a5-4ae3-a51d-e4da99f1634b None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.435462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05bc8c-e0a5-4ae3-a51d-e4da99f1634b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.435735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05bc8c-e0a5-4ae3-a51d-e4da99f1634b None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.438769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 387/771] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:36.444238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.444540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.448297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.448428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 385/772] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:36.450626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:36.485451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:36.486940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:36.515057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:36.530753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:36.606194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41cb8-5cae-4f48-98de-0ba7dfc33e3b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:36.649505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] RBAC: Adding query filter params (interface=public, service_id=78628698be334f1eaa047aabfca74c65, region_id=RegionOne) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.655084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f66f56-504b-4e5d-af15-c459e9f45bce None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.656275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 388/773] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:36.668645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 386/774] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:16:36 2022] GET /identity/v3/endpoints?service_id=78628698be334f1eaa047aabfca74c65&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:36.675458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:36.771188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6bd1e-408e-4e02-a26c-e4ecac995615 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.771736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6bd1e-408e-4e02-a26c-e4ecac995615 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.771736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6bd1e-408e-4e02-a26c-e4ecac995615 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.773347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 387/775] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:36.778191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.778470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.778743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.798417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:36.799520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:36.808403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cd822-b23a-4fd4-94f6-801b366c208a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:36.873315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:36.874057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:36.874341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:36.874666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:36.875277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:36.876670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:36.885977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 389/776] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:16:36 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:36.899715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 388/777] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:37.002839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.028030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.142789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ac26-68c5-4634-a9bb-762d78c7ac29 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.178847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.179482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.179754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.180022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.191329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:37.196296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:37.197258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:37.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 390/778] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:37.205649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.227837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.269913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.332630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b099d-afc8-4da3-8e97-787dd4b43f82 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.390498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.391042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.391316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.391582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.392271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:37.392569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.393925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736725-a0a8-4cea-adb9-dc721e082747 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.403862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 389/779] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:37.404629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.406955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19377a-9a76-4ccd-8624-fb1b692d917e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.407532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19377a-9a76-4ccd-8624-fb1b692d917e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.408014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19377a-9a76-4ccd-8624-fb1b692d917e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.413828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 390/780] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:37.413828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19377a-9a76-4ccd-8624-fb1b692d917e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.436448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6720d4-feeb-406e-925d-f19d2d182cae None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.484771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.485413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.485689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.485969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.486801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.488465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2c49cfe8-ba5a-470b-bcd4-877fa8a0194a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:37.501385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:37.502415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:37.503686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 391/781] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:37.504543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.504543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.508538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.523816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.535776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:37.537674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:37.548851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19377a-9a76-4ccd-8624-fb1b692d917e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.590872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.591930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.592476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.593072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.604732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.607311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:37.626188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:37.627185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:37.628099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:37.628099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:37.628099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:37.630311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 391/782] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:37.630422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.638815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 392/783] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:37.638995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.720710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.756770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2a389-5b4f-493e-9fc7-6a2ba4fa315a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.758985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6808-3229-4d3d-94a7-82943fbf3327 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.794782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.795406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.795718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.796167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.796830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:37.797127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.798491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.803676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 392/784] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:37.804688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:37.809283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.828974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.829750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.830088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.830425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.831233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:37.831594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.841671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.860206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 393/785] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:37.865827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:37.927593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bd049-2fde-4b2b-8e80-d88cfea906de None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:37.978595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:37.979494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:37.979883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:37.980293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:37.981178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:37.981480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:37.983941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:37.990334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 393/786] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:37.991171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.045444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01018a8-4ba5-45a8-bcca-5ef644b47aa2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.076561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.078312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.088882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.089965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.090443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.090911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.100719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.102522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:38.121096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:38.122253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:38.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:38.125323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 394/787] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:38.126281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.133888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de2fc1-3a22-4133-ba4d-3b8a85350055 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.172367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.173331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.173742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.174167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.185654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:38.192378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:38.193503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:38.194392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:38.202458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 394/788] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:38.203472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.242909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59df0-aa54-44e3-ae36-aa305868d026 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.281270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.282016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.282755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.282755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.283662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] RBAC: Adding query filter params (interface=public, service_id=88bea3c48cbf411f8437bc6d30859ee2, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:38.284146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.285715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.293869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 395/789] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/endpoints?service_id=88bea3c48cbf411f8437bc6d30859ee2&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:38.300107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.320377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5c8b-da3e-4c75-989e-8d5db12c8d36 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.348046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.361645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.362554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.363043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.363487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.364414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:38.364893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.366654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.368055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.390450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 395/790] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:38.392062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.419890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe977d-7c95-43d6-8a33-9cc0384ad7f0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.470289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.471028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.471774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.472127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.473160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:38.473536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.474991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.491112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 396/791] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:38.491112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.525335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c92627-283f-4a53-b541-fddd460164c0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.566715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.567787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.568288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.568730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.569499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.571299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.578563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 396/792] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:38.580141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.604845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43a0d4-8c7d-4225-9a48-1fa050f68fbe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.625390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.645410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.645628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.646450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.646562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.646886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.659591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.661279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:38.677187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:38.678193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:38.679084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:38.679084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:38.679084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:38.679084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 397/793] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:38.689540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.716767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a7ef-e911-4bd6-a175-231b80d04628 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.759766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.761927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.762442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.762915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.776887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.778713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:38.796947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:38.798372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:38.799249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:38.799249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:38.799249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:38.801981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 397/794] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:38.808923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.868192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961f01-e054-4adf-8338-b260e29a4b9b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:38.916206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:38.916928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:38.917208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:38.917482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:38.918465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:38.918768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:38.922038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:38.939862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 398/795] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:38.945322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:38.945955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:38.953191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d541b-0786-4c25-ba5a-529b6b3615a8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.000154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.003862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.004148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.004447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.005101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:39.005405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.007787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.023226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.034584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 398/796] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:39.041238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.112720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb7c46-9b81-4a5a-873b-cdcff840fc90 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.168102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237fc72-1f36-4bd6-8f32-b9acd03c5bf0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.170246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.171034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.171312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.171582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.180526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.182454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f86b0437-88fc-4a51-a100-357a799b2e83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:39.197925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:39.198897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:39.199819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:39.204322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 399/797] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:39.207317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a461-fc4d-4ea1-8846-8dba4539979d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.207951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a461-fc4d-4ea1-8846-8dba4539979d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.209425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a461-fc4d-4ea1-8846-8dba4539979d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.215548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 400/798] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:39.215548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a461-fc4d-4ea1-8846-8dba4539979d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.222830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.223463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.223739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.224005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.232821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.234518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-41039362-b7ea-41cd-93d2-fcd1767754b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:39.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:39.252104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:39.253230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:39.254929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 399/799] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:39.256394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.256756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.257380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.279819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:39.282277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:39.299897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.328049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.341241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a461-fc4d-4ea1-8846-8dba4539979d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.384283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.384514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 400/800] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:39.384514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.385623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.386100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.386564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.387588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:39.388086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.390100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.415840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 401/801] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:39.422376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.505768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949c0fd-2341-4c2e-8ad7-d8b316f029fb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.545079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.546074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.546355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.546628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.547728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:39.548035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.549763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b86c-fdfb-43c5-a491-a7581737af42 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.576427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 401/802] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:39.576427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e3a5-2141-4c1a-9423-36d46ecb42b3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.576427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e3a5-2141-4c1a-9423-36d46ecb42b3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.576427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e3a5-2141-4c1a-9423-36d46ecb42b3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.576427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 402/803] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:39.578510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e3a5-2141-4c1a-9423-36d46ecb42b3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.583197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.609887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede23a7-a19f-44de-b9f8-9e933bffd49a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.654276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.656000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.656636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.657011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/85e1f5c17df9495182082a9d85fa3401/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.676672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=85e1f5c17df9495182082a9d85fa3401, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.678268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6eb4df-5b8e-44de-8847-6fd7128170d7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.689355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.703537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 402/804] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:39 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/85e1f5c17df9495182082a9d85fa3401/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:39.703537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.703537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.703537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.725112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:39.726761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:39.735535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e3a5-2141-4c1a-9423-36d46ecb42b3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.781497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:39.782140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:39.782412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:39.782744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:39.796332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:39.798704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:39.818538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:39.819727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:39.820740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:39.820740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:39.820740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:39.820740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 403/805] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:39.820966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.830242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 403/806] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:39.831038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:39.937847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:39.988810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59cfe6-f797-4938-8569-126688fe871b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:39.992276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541de2-3921-4a65-9faa-b6bf3c8dccb6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.028700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.029385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.029663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.029937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.036636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.043192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:40.043607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.044328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.044676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.045018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.045782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:40.046363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.047972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:40.049800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:40.050921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:40.051946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 404/807] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:40.065954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.067127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 404/808] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:40.078592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.251322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a739f-f220-41b7-a8a7-f8d3060ecd79 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.292579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.293658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.294198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.294472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.302675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce016f-a9d9-4996-af16-cb850cc60a9d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.303729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.306397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.319770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:40.322254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:40.323202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:40.324100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:40.329339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 405/809] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:40.329339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.345387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.346054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.346335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.346604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.347261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:40.347557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.348929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.358210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 405/810] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:40.359443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.402889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.459819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1de31-9e80-41d2-abfe-62ad7d69ccdc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.471325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cd608-22f0-4050-8450-091055982f4d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.497310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.498255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.499315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.499768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.500579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.502126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.509120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.509717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.509988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.510253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.510925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:40.511221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.512631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:40.512821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:40.513772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:16:40.514659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:40.517390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 406/811] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:40.518448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.530698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 406/812] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:40.531492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.573136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.637208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95ce1-34a6-4d4a-ae28-d59b8d0b09e9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.653022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.655986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f39b-7bc7-4c24-b546-462be4935a29 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.676419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.676821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.677099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.677371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.699735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.700717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.701139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.701562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.703013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] RBAC: Adding query filter params (name=network) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:40.703476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.705057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.710958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 407/813] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:40.712038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.815271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.816479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.830277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942307-b95b-4623-a845-098553a96b45 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:40.832051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:40.833182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:40.834179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:40.834179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:40.834179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:40.834179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 407/814] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:40.834416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.834916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:40.879167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:40.880103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:40.880992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:40.881437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/ed5f73bac406439e856d745885c0c092/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:40.884170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=ed5f73bac406439e856d745885c0c092, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:40.886518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:40.912611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 408/815] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:40 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/ed5f73bac406439e856d745885c0c092/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:16:40.912611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:40.987444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26621b-f22e-40bb-af7c-ff9061788c84 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.031079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.031705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.031979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.032276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.032938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:41.033297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.034851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.041581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 408/816] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:41.048142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:41.094828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:41.144770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2b5a-6f60-4866-a3bf-35da08bccb8b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.186754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.187535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.187817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.188086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.188913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:41.189214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.190575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef6ae-c726-4070-9ed8-43d152b05cbd None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.208654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 409/817] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:41.209982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a1a54-37db-42c6-a59d-bedf34372c4c None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.210263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a1a54-37db-42c6-a59d-bedf34372c4c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.210582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a1a54-37db-42c6-a59d-bedf34372c4c None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.212056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 410/818] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:41.218142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a1a54-37db-42c6-a59d-bedf34372c4c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:41.239303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1e926-a373-44dc-91dd-8bbf454fbaf9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.250063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:41.287411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.289291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.289769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.290500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.291608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.293747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f41e21-47aa-4e85-934e-5cd4626a1c24 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.330031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 409/819] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:16:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:41.331282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.331563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.331834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.353853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:41.355779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:41.415821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a1a54-37db-42c6-a59d-bedf34372c4c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.449529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 410/820] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:41.472913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.473567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.473842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.474110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.482812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.485225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:41.499787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:41.501030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:41.501994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 411/821] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:16:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:41.507022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:41.527953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:41.617741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e62dd-bc52-4d50-a681-ff416da0e13a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.653665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.654405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.654642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.654945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.655728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:41.656028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.657575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.676599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 411/822] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:16:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:41.682137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:41.768908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:41.792293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c759d0-25eb-4d02-90fb-f1b36591cf0a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:41.825297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:41.831842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:41.832539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:41.832790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:41.833065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:41.921394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:41.922852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:41.928838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:41.965359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ddeb9-0f14-4706-8687-95943c453af1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.005375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.006373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.006892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.007381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.009109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.010738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.011601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:42.024021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:42.025097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:16:42.025949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:42.028922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 412/823] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 05:16:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:42.035812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.040319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 412/824] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:16:42.041853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.158524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7ddf6-5483-404e-8d69-4d826ff71cc2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.182775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b23a3-cc9a-4850-93ed-1b27c764fc53 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.195615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.196575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.197062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.197536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.198364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:42.198902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.200492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.210999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 413/825] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:42.212201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.220403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.224736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.225371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.225672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.225913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.238455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.239811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-20d19924-3b26-4ba5-9f1c-a0eb540588d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:42.259231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:42.260817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:42.262012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:42.262012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:42.262012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:42.262012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 413/826] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:42.266007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03efac-38cd-4a9b-be4a-260a41d36221 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.266291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03efac-38cd-4a9b-be4a-260a41d36221 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.266562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03efac-38cd-4a9b-be4a-260a41d36221 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.267979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 414/827] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:42.268568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03efac-38cd-4a9b-be4a-260a41d36221 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.301369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.332545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36826cd5-5a91-4e96-8dd3-5155ed64ce10 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.365361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.366108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.366380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.366645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.367258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:42.367553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.369185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7742e-9ca3-47f5-8b21-f8b05cb6ca85 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.379657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 414/828] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:42.379657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.379657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.379657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.381802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03efac-38cd-4a9b-be4a-260a41d36221 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.395644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:42.396836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:42.423993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.424590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.424869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.425141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.425772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:42.426074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.427431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.444784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 415/829] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:42.444784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.460385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.469837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 415/830] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:42.470851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.561369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.573567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0492ea-be0c-4e43-a4f9-c7e402e70108 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.598742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeb7a0-2326-46ae-a295-b4ab06baca74 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.613098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.613934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.614364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.614825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.615729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.617762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.641422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.642240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.642713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.643084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.644475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 416/831] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:16:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:42.655734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.655734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:42.669543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:42.671060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:42.672074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:42.676611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 416/832] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:42.678243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.789373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.806936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c66701-8367-4d31-8277-1d4d2df17022 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.849751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.849751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.850426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.850426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.850836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:42.851136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:42.853360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:42.857331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.866886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 417/833] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:42.873570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.876497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:42.909463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8951cd-1c7a-4c7f-96a6-c1c3a3cb3ca8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:42.946886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:42.947587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:42.947859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:42.948169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:42.959459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:42.967223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:42.968771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:42.969638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 417/834] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:42.974970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:42.987659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cdf-278c-4c09-97dd-fa80e04d2b0d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.028220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.029121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.029395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.029665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.030856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:43.031168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.032620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.040026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 418/835] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:43.096112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef91b-62b3-409d-b73c-826a729e2423 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.134366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:43.138051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.139159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.140212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.140693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.141376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:43.141677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.143082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a066b20-f517-4254-926f-f9d5ca4d4230 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.153068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 418/836] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:16:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:43.159157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.238841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:43.276919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7b48-1ef7-4775-ac02-f64f0cc54f41 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.297093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a67f2-1635-4165-8130-46e9294b387a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.297410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a67f2-1635-4165-8130-46e9294b387a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.297642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a67f2-1635-4165-8130-46e9294b387a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.299235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 419/837] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:43.304355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.304635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.304908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.317402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.318032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.318305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.318571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.326550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:43.328827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:43.331916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.334471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:43.351973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:43.353206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:43.354310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:43.354310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:43.354310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:43.355687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 419/838] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:43.364538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.419934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 420/839] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:43.483676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae75e5-cfa2-4209-bcaa-7777708fa3cc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.502505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:43.520457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.521371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.521828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.522276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.523205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:43.523956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.525531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.538579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 420/840] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:43.544294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.651534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50503-e445-4070-a225-6b409dd1c31a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.662279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:43.688277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.688908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.689180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.689446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.692275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.697113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.698551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:43.711654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:43.712709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:43.713756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:43.717301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 421/841] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:43.719054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.806869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939fb9-7024-4930-9b46-1c2b0099bca0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.824355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9be1-1715-412a-8ee7-ed4fba557ddf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:43.845177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.846301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.846825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.847304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.859437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:43.861067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:43.861671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:43.861944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:43.862210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:43.862924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:43.863222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:43.864671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:43.866960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:43.867887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:16:43.868786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:43.871735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 421/842] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:43.878927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.883529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 422/843] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:43.888988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:43.919978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:43.935474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.001121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cfc27-01a3-4aa8-aea4-832e7995c9fc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.018291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceffd6-8de6-4602-8710-7b9686f38885 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.037370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.038054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.038257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.038564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.063411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.064571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.065433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.066131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.067367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:44.068413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.070491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.083075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 422/844] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:44.088993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.129547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.131890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.164451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.205607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25840d-53ed-426a-8ad6-bf605759ddd8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.208661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.247458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.248366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 423/845] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:16:44.249254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.249838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.250253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.261563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.262197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.264095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:44.283464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:44.284596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:44.285503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:44.285503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:44.285503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:44.288616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 423/846] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:44.291491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.391286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9741c-b8ee-4926-ad24-84a323b59ed1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.423993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.434506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735b0b4-025c-42b5-a0e0-7438a0172354 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.441571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.442553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.443542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.444059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.459280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.461354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.478414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.479315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.479800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.480201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:44.481269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:44.482169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:44.483181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:44.483181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:44.483181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:44.483522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:44.483522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.485680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 424/847] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:44.488116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.492437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.505110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 424/848] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:44.512863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.532815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.617033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5255180-1a61-47eb-8547-6d6665a4866d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.637551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9783a-033e-43d9-9c50-7f52a833a532 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.656039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.656729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.657007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.657277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.657904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:44.658204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.659666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.676366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 425/849] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:44.677274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.683157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.684016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.684544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.685012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.696026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.698412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:44.715868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:44.717090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:16:44.718068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:44.720552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.722136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 425/850] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:44.728673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.805877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a506-3fbc-439e-a4ba-5350022f4840 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.825253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:44.865878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6094bcb-5955-4d8d-9536-0ce83128beca None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:44.874933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.876134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.876740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.877641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.892577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.895551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.908337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:44.909222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:44.909671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:44.910111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:44.911011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:44.911503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:44.913096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:44.921259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:44.922141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:44.923045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:44.930370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 426/851] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:44.932851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:44.934926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 426/852] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:44.942369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.043816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.049202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11e8ac-3f56-4f3c-9627-f8d21ceca607 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.053570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef10b-8e09-4f4c-8168-79e138acce8b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.085905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.089967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.090583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.090862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.091136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.091828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:45.092134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.092982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.093659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.093819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9a0d0-fbdd-4e7e-b7f9-9cbb7742e53e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.093921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.094211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.107621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 427/853] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:45.110326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e4c62-462e-448c-a7fb-17e090808aaf None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.110812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e4c62-462e-448c-a7fb-17e090808aaf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.111280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e4c62-462e-448c-a7fb-17e090808aaf None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.112982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 428/854] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:45.113905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e4c62-462e-448c-a7fb-17e090808aaf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.173777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.175243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.221084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e4c62-462e-448c-a7fb-17e090808aaf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.254892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.255497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.255775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.256046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.256794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=85e1f5c17df9495182082a9d85fa3401) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:45.257101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.258537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72bac-af52-4474-8871-d1dd1ff98e06 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.264905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 429/855] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/role_assignments?user.id=85e1f5c17df9495182082a9d85fa3401&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:45.266051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ff9f-42df-4469-9d30-e6864aefdddf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.266334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ff9f-42df-4469-9d30-e6864aefdddf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.266607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ff9f-42df-4469-9d30-e6864aefdddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.287811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7a20ff9f-42df-4469-9d30-e6864aefdddf None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:45.289068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7a20ff9f-42df-4469-9d30-e6864aefdddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:45.300677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 427/856] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:16:45.306995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.308343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.353654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.391797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 430/857] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:45.436612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db38dc-c286-4c8c-8d71-216a29a13c81 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.470191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed1b71-14cf-4202-b033-f412756e5f00 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.470191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed1b71-14cf-4202-b033-f412756e5f00 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.470778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed1b71-14cf-4202-b033-f412756e5f00 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.472139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 431/858] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:45.475203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.476077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.476584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.477061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.477268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.477498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.477773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.490413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.491853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.502984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:45.504185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:45.513011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:45.513959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:45.515190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:45.515190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:45.515190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:45.515190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 428/859] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:45.520624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.597723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 432/860] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:45.621755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.641690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4aa35-e4c6-4c75-a866-107f781994c4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.667875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.676371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.676981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.677254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.677519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.678155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:45.678450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.679793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.696364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 429/861] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:45.697891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.774733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.799865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea9890-e65d-4818-b7fd-649131456f85 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.807922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4088b-14d6-487c-8393-2f279f6fc09d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.839750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.840632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.841019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.841383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.842053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.844354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.849094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:45.849664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:45.849943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:45.850217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:45.854348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:45.855556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:45.856487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:45.857252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:45.857328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 433/862] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:45.860289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:45.865693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:45.876072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:45.877413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:45.878305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:45.880112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 430/863] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:45.881095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:45.883980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:45.978981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95d34-618d-455f-833a-6382708a5b8a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:45.993180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d95c3-297f-465d-b950-6960a3ed0985 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.016694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.017303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.017581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.017854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.018479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.018780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.020145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.029483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 434/864] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:46.029483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:46.034228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.034875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.035155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.035427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.036013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.037403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.041255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:46.047038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:46.048012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:46.048981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 431/865] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 05:16:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:46.049808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:46.123615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:46.148769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c05295-d71f-4ad0-b697-da1b3672d562 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.163235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24ebd7-345c-4792-9913-cd797965b5c2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.185407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.186179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.186616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.187081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.187776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.188085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.189583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.200549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.201156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.201435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.201731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.202353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.207525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.207656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 435/866] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:46.207656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:46.207863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.211875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 432/867] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:46.220378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:46.299424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:46.347389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a838aa8-666e-4479-8644-85b39623487e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.359324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d66708-e438-4c50-a7b6-9559a88f629b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.397066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.398047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.398509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.398974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.399791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.400333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.400731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:46.402526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.408726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.409347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.409629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.409905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.410637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=ed5f73bac406439e856d745885c0c092) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.410943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.411739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 436/868] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:46.413267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dabc3-d207-4ada-b742-f983cef29fa3 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.418327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:46.418774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 433/869] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:46 2022] GET /identity/v3/role_assignments?user.id=ed5f73bac406439e856d745885c0c092&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:46.541689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603bd4-1a07-4670-b300-0963996a4139 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:46.582709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:46.583333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:46.583614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:46.583888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:46.584562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:46.584862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:46.586343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46365d4d-ccec-4c15-94e7-0b04cb32efd0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:46.587769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:46.592833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 437/870] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:47.640925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230db9-96f6-42d6-9673-d71839026205 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:47.641451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230db9-96f6-42d6-9673-d71839026205 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:47.641451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230db9-96f6-42d6-9673-d71839026205 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:47.643009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 434/871] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:47.648421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:47.648811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:47.648973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:47.667356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:47.668554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:47.740848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 438/872] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:48.013012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e230db9-96f6-42d6-9673-d71839026205 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.118632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e230db9-96f6-42d6-9673-d71839026205 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:48.157201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:48.157838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.158120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.158394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.169557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:48.175342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:48.176278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:48.177139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 435/873] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:48.182441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.288712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93a81-d982-4fec-b43b-db9fa431558a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:48.318928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:48.328036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:48.329036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.329512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.329967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.330702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:48.331004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:48.332456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee487-eebf-41ae-817b-3cb9e00699ce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:48.341390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 439/874] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:48.346822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.409546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:48.456696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f238-7b19-4b07-a421-a8f8f436bc2f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:48.491570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:48.492280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.492559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.492876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.497824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e93237-d5a4-4317-9dc5-a63e1571400b None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.498099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e93237-d5a4-4317-9dc5-a63e1571400b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.498902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e93237-d5a4-4317-9dc5-a63e1571400b None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.500661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 440/875] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:48.505810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.506022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.506348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.507270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:48.508959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:48.524160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:48.525160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:48.526050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:48.526050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:48.526050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:48.527611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:48.528424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 436/876] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:48.533919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:48.537247 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.607952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 441/877] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:48.647690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7fc-385b-49d5-bfec-54979a33cf8d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:48.656439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:48.684709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:48.685319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.685599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.686021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.686772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:48.687078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:48.688467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff02cf-bc6d-40f1-9395-6aa3e36263e5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:48.700892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 437/878] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:48.707440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.757465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ee86-2bea-4ffe-b986-9d185e6cab06 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.757788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ee86-2bea-4ffe-b986-9d185e6cab06 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.758020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ee86-2bea-4ffe-b986-9d185e6cab06 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.759496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 438/879] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:48.765599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.765885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.766161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.789902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:48.790645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:48.823387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619cca8-5011-4214-9c93-e3e2a5c2aebd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:48.844926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:48.861015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:48.862097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:48.862637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:48.863123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:48.876276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 439/880] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:48.877687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:48.878504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.879183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:48.895708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:48.896773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:48.898193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 442/881] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:48.904796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:48.984912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:48.992095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b140d-efb6-4532-9df9-8ab92484c99d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.013669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597cc0-0711-4031-959a-4156a67816a0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.028166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.028878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.029161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.029969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.030661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.032167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b8178-b39c-4ae0-ad2c-a1c09b72608a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.047706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 440/882] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:16:48 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:16:49.053740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe751f-2ad4-456b-b81c-f376ed377f1c None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.054064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe751f-2ad4-456b-b81c-f376ed377f1c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.054287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe751f-2ad4-456b-b81c-f376ed377f1c None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.055644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 441/883] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:49.061828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.062164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.062390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.066338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.067414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.067893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.068420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.070823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:49.071446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.073185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.083114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:49.084543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:49.099321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 443/884] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:49.104699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.157368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.175806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 442/885] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:49.177528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.239403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf4cd-ebea-4ac1-a1b1-198e515b156e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.288967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.289172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.289512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.289726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.323231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.346037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd87d6f-7ebb-4ff4-8a3e-78db4db77ce3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.370280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.371825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.392002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.393073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.393352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.393625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.394213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.396116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:49.408559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:49.409538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:49.410377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:49.411113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.411186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 443/886] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:49.417556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.491988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 444/887] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:16:49.492856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.541621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40878d59-1f6e-4d26-aa93-505e17407975 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.576837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.577422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.577695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.577962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.578581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:49.578877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.580220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.585084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 444/888] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:49.590324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.596191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e02d9-b9dd-41c3-b589-bbb1903760dd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.608605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.633760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.634369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.634640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.634912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.635554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.637228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.641177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.660396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 445/889] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:16:49 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:49.660396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.719261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068faa-0b4e-4f17-bda0-634c9e625d26 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.771023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.771672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.771952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.772228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.784802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.786269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:49.804241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:49.805496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:49.806690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:49.806690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:49.806690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:49.810170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 445/890] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:49.811494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:49.822657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9650a-e8f5-45c8-831e-37cd2c933612 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.872661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.874324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.874982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.875441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.876275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:49.876761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.878328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:49.883143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 446/891] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:49.892669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:49.944404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8eef9-4667-4c16-92a3-92f2915fa6c6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:49.986873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:49.987883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:49.988333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:49.988772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:49.989567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:49.990007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:49.991523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:50.006423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 446/892] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:50.013469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:50.083954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:50.133826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d090ad2-9ba8-4696-b70d-47e8becd6838 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:50.170144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:50.170840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:50.171122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:50.171395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:50.178684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:50.180127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:50.191673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:50.192614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:50.193897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 447/893] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:50.199020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:50.307548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d05b-d37b-4ad5-ab91-638ffe926c05 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:50.331425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:50.343539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748528b0-461d-4961-9d70-31b764763070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:50.344155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748528b0-461d-4961-9d70-31b764763070 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:50.344456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748528b0-461d-4961-9d70-31b764763070 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:50.344730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748528b0-461d-4961-9d70-31b764763070 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:50.345428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748528b0-461d-4961-9d70-31b764763070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:50.345733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748528b0-461d-4961-9d70-31b764763070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:50.347151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748528b0-461d-4961-9d70-31b764763070 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:50.360382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 447/894] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:50.366269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:50.472369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f29c2-48f3-4aa9-a2cd-9551eef67206 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:50.489261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:50.507242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:50.507844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:50.508168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:50.508539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:50.509310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] RBAC: Adding query filter params (role.id=0046b1387736483ea5f7ccb93610540d, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=85e1f5c17df9495182082a9d85fa3401) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:50.509618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:50.511054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9d5c-e860-4a93-8b29-9c27cdf4b13d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:50.517465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 448/895] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:16:50 2022] GET /identity/v3/role_assignments?user.id=85e1f5c17df9495182082a9d85fa3401&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=0046b1387736483ea5f7ccb93610540d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:51.304947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363571-455a-4789-a2fa-d533f6d7c822 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:51.305460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363571-455a-4789-a2fa-d533f6d7c822 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:51.305460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363571-455a-4789-a2fa-d533f6d7c822 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:51.307018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 448/896] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:51.312462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:51.312881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:51.313051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:51.330936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:51.332107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:51.406720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 449/897] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:51.671175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12363571-455a-4789-a2fa-d533f6d7c822 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:51.776864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12363571-455a-4789-a2fa-d533f6d7c822 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:51.816441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:51.817070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:51.817348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:51.817620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:51.818194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:51.819565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:51.828902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:51.829919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:51.830761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 449/898] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:51.835821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:51.944952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ae44-13b7-4a92-a7d8-dae176cb1a34 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:51.960777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:51.985456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:51.986076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:51.986348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:51.986615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:51.987246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:51.987540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:51.988925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df7bb-fa06-4d08-be35-03ccd9245b7d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:51.994349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 450/899] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:52.001315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.050293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:52.085824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04406af7-01b4-487c-b452-1437eeb286dd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.086167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04406af7-01b4-487c-b452-1437eeb286dd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.086371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04406af7-01b4-487c-b452-1437eeb286dd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.087997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 451/900] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:52.094370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd1b9e-c0a3-4334-9e7d-775dac6ceb10 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.094673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd1b9e-c0a3-4334-9e7d-775dac6ceb10 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.094918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd1b9e-c0a3-4334-9e7d-775dac6ceb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.116040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878cb7c-a58a-4b82-9b1b-ac78b6b5b4ab None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.121240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a5dd1b9e-c0a3-4334-9e7d-775dac6ceb10 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:52.122686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5dd1b9e-c0a3-4334-9e7d-775dac6ceb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:52.156408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.157173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.157545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.157916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.158649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:52.159051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.160852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.171060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 450/901] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:52.179771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.208073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 452/902] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:52.209650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c2848-08f7-4657-a6c3-ada8129bfe52 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.210132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c2848-08f7-4657-a6c3-ada8129bfe52 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.210623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c2848-08f7-4657-a6c3-ada8129bfe52 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.212213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 453/903] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:52.222474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.222474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.222474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.245697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:52.247752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:52.301593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a004e5-81a8-4e7d-a21a-c96d4a3316ab None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.314347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:52.323518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 454/904] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:52.346819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.347465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.347742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.348008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.348793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] RBAC: Adding query filter params (interface=public, service_id=bf0840dbbf0540448b00e662c2f460bd, region_id=RegionOne) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:52.349093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.350485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.356015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 451/905] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/endpoints?service_id=bf0840dbbf0540448b00e662c2f460bd&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:52.362411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.412590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:52.465052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2918bf9-001c-4741-9991-3769742b9364 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.489547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.501397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.502112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.502386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.502654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.503408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.504915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.511252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 455/906] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:16:52.613226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92dbcf-122e-437e-9d21-e97fc780c311 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.633771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.657030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.657615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.657899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.658174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.658770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.660161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:52.670450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:52.671487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:52.674303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 452/907] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:52.677729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.725073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:52.760945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:52.765491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4f788-5822-429b-8cc9-7824d5c8500f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.791116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5f64-5b13-4dda-84f7-13cd2a60ae11 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:52.803307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.803925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.804202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.804504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.805157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.806574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746a297-5a3c-4735-a5bf-8569a0f47809 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.824293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 456/908] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:16:52 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:16:52.825323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d60c1-b3aa-4bc1-b4cc-1cfaf0b33cee None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.825607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d60c1-b3aa-4bc1-b4cc-1cfaf0b33cee None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.825880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d60c1-b3aa-4bc1-b4cc-1cfaf0b33cee None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.832374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 457/909] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:52.833659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.833943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.834957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.847108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:52.847734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:52.848014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:52.848309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:52.848957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:52.849262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:52.850751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:52.856636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 453/910] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:52.861844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:52.863266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:52.865360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:52.953555 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 458/911] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:52.998297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:53.002152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae61-0d53-4ecd-920f-967ed79447c5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.037357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.037970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.038246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.038513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.039141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:53.039438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.040814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.046599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 454/912] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:53.052160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.151954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:53.162688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd5355-3d39-4ab6-9113-4f6203bf6260 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.199965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.200610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.201008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.201899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.202791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] RBAC: Adding query filter params (interface=public, service_id=4641cfd3e51648cc9d1ae7c1121869a8, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:53.203101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.204503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.210014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 459/913] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/endpoints?service_id=4641cfd3e51648cc9d1ae7c1121869a8&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:53.215548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.266413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.324661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e0d6f-5a60-48a0-81e6-74dc995fb151 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.361437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.362045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.362325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.362596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.363195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.364591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.376391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 455/914] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:16:53.383843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb61ea-7d7a-4e8d-a6cc-302ff8c946bb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.427255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.428357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.428842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.429343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.448283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:53.449488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:53.450400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:53.452403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 460/915] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:53.461431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.495653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:53.577124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829526bf-a52d-4e23-9164-9b8c26564b47 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.612547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.613524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.613802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.614067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.614795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:53.615098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.616459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.625820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 456/916] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:53.631438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.652971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:53.742301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8262b8-5fb1-4add-ae48-6f10be3c3588 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.779785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.780451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.780720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.780987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.791867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.793453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:53.809188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:53.810180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:53.812110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:53.812110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:53.812110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:53.812110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 461/917] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:53.818411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:53.922094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:53.933848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ada89-22d8-465f-99ab-05ebdbf1393d None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:53.969018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:53.969640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:53.969916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:53.970184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:53.970817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:53.971112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:53.972514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:53.985612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 457/918] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:53.991359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:54.059313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:54.097241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa134704-11b1-4a68-8f58-11c42043127b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:54.133247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:54.133885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.134160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.134433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.142804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:54.144280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:54.155780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:54.157107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:54.158242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 462/919] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:54.164925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:54.270434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9306ab-a4f7-440b-92f8-116c2ef68c9b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:54.307232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:54.308214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.308734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.309213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.310133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:54.311180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:54.312831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:54.336378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 458/920] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:54.341312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:54.452608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615180c5-8eed-49b7-9413-65e5a931b68f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:54.478377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:54.487608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:54.488234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.488550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.488823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.500312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:54.501747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:54.516481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:54.517423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:54.518775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:54.518775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:54.518775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:54.518775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 463/921] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:54.525024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:54.536616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:54.634118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1e7e2-2b05-400b-9c91-9c9e5d26f25b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:54.670842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:54.671789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.672457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.672956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.673779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:54.674274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:54.675807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fb145-7e13-475e-ae8d-7d5dbfea79b9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:54.692295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 459/922] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:54.694200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:54.727461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de29-d6db-4127-8c8d-a2aef1c6ec30 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.727769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de29-d6db-4127-8c8d-a2aef1c6ec30 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.728009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de29-d6db-4127-8c8d-a2aef1c6ec30 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.729614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 460/923] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:54.735793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.736065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.736371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.755710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:54.756965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:54.822491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:54.822853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b65f0-8574-4f07-897c-eb915b643cf1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:54.832294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 461/924] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:54.865627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:54.866608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:54.867061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:54.867519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:54.875894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:54.877597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:54.889369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:54.890308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:54.891199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:54.893352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 464/925] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:54.899009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.002250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09e4bd-960f-473e-a516-0dac5a019dbc None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.036974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.037926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.038388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.038822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.039690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:55.040164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.041767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.054553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 462/926] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:55.062012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.104798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.106438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.176014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befda95-2421-4fc6-90d3-7db80b086736 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.187913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.219911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.220894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.221338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.221758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/85e1f5c17df9495182082a9d85fa3401/roles/0046b1387736483ea5f7ccb93610540d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.224311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=85e1f5c17df9495182082a9d85fa3401, role_id=0046b1387736483ea5f7ccb93610540d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.225995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1d66e-6386-4c33-945c-691d8c613dc2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.251135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036cf6-186a-4592-94c9-27c76deaf780 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.251339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 465/927] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:16:55 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/85e1f5c17df9495182082a9d85fa3401/roles/0046b1387736483ea5f7ccb93610540d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:16:55.289170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.289817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.290091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.290356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.290950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.292359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.294608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35cae-45aa-4e49-a029-9b62726fbfe8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.294918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35cae-45aa-4e49-a029-9b62726fbfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.295273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35cae-45aa-4e49-a029-9b62726fbfe8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.297156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 466/928] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:55.306400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:55.307526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:55.308454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 463/929] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:55.313071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.313672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.314154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.314310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.337493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:55.338847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:55.428657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 467/930] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:55.443611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090010b-ec56-4fd0-8f34-d70acdcf5781 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.464446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.480153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.480785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.481058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.481324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.481942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:55.482757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.484444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.489466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 464/931] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:55.495214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.524070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.605360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b11348-7ea1-42a5-bac5-7ddb2d261d2a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.646213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.646848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.647205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.647485 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.648357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:55.648847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.650274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.657149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 468/932] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 05:16:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:55.662456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.713104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.772882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19fb6-949f-4fe5-b324-ec08a6e3d00b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.808839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.809560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.809844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.810114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.811340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.812742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5cb8f-01ec-4454-91ac-83145d8d4e60 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.814772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.824611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304df3-8530-44b5-bdd3-f88d83aaaec5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:55.836581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 465/933] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:16:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:16:55.837908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d6db8-c0b0-4cf4-9d58-3022ceb46648 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.838265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d6db8-c0b0-4cf4-9d58-3022ceb46648 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.838638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d6db8-c0b0-4cf4-9d58-3022ceb46648 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.840233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 466/934] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:55.845956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.846261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.846522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.866757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:55.867993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:55.905723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:55.907283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:55.907558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:55.907828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:55.908459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:55.909888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:55.915384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:55.922067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:55.923150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:55.924007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 469/935] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:55.930302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:55.967940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 467/936] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:56.055805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d272cc-4c60-4b03-96b6-5ea326730762 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.100294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.100913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.101192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.101463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.102092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:56.102446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.103823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.112314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 470/937] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:16:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:56.115447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.222575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946439-3631-4629-a8a2-cba1e9a6b157 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.237269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.262168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:56.265842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.266724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.267200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.267692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.268839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:56.269749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.271611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.278118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 468/938] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:56.292311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.322443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:56.358169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8612d-3c80-4959-9ec1-eec715653cf5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.400169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf59ea-375e-4610-a12e-f835bcb0e650 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.405917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.406557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.406831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.407096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.407672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.409081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:56.421127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:56.422098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:56.422983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 471/939] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:56.427875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.443450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.444391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.445312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.445801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.446697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] RBAC: Adding query filter params (interface=public, service_id=f019fce1560c40e688377a209fcc68a2, region_id=RegionOne) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:56.447116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.448641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.459490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 469/940] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/endpoints?service_id=f019fce1560c40e688377a209fcc68a2&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:56.461602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.524291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:56.547733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c88833-4f24-4a0e-be06-43de6a2605c5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.574893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:56.580797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b3c9b-aed9-4115-89c3-168c3037fa31 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.585206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.585938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.586311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.586674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.587396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:56.587789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.589393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.600482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 472/941] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:56.601637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.625224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ec54-435a-449c-86b7-512010906153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.626618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ec54-435a-449c-86b7-512010906153 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.627070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ec54-435a-449c-86b7-512010906153 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.627485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ec54-435a-449c-86b7-512010906153 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.628538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103ec54-435a-449c-86b7-512010906153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.630286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103ec54-435a-449c-86b7-512010906153 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.637981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 470/942] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:16:56.725449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297aa71-5fad-46d8-823d-3dc0da5b90fe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.766325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.767197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.767616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.768027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.768868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:56.769315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.770961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33869-a352-4a39-ba99-a421b2ce49f1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.777474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 473/943] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:56.783757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ec54-435a-449c-86b7-512010906153 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:56.892401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ec54-435a-449c-86b7-512010906153 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:56.927682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:56.928672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:56.929148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:56.929609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:56.930504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:56.932043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2965d6-c629-4d08-99f6-63bab7fc3aeb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:56.956138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 471/944] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:16:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:16:57.599582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203314f3-c6be-43b7-8250-da91be4e512a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:57.600162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203314f3-c6be-43b7-8250-da91be4e512a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:57.600162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203314f3-c6be-43b7-8250-da91be4e512a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:57.601730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 474/945] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:57.607113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:57.608489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:57.608959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:57.628532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:57.630023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:57.644941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:57.709124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 472/946] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:57.987415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203314f3-c6be-43b7-8250-da91be4e512a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.000740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.104532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203314f3-c6be-43b7-8250-da91be4e512a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.140627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.141255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.141531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.141799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.153261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-5ac8f753-f430-4c1e-9f3c-4bcf7f5c2512 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:16:58.158162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:58.159085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.160211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 475/947] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:16:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:58.165238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.173695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281354-ed56-4616-b469-7def41199f82 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.173982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281354-ed56-4616-b469-7def41199f82 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.174232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281354-ed56-4616-b469-7def41199f82 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.175681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 476/948] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:16:58.180721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.181199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.181652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.203919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:58.205374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:58.277908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.292614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 477/949] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:58.302866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f93d86-8802-4f4e-a245-b7e180f51480 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.339804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.340439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.340767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.341038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.341722 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:58.342018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.343436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:58.352721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 473/950] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:16:58.361383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.472300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3dfe-ad91-419e-a345-229765875fbf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.507948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.508598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.508980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.509252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.519633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.521103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:58.536081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:58.537095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:58.538042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:58.538042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:58.538042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.538042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 478/951] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:58.543367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.553449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.585101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.646346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.663673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff006cdb-fcca-44b4-a87c-dd2cf94d95e4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.704924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.705696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.706068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.706453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.707304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:58.707707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.709210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:58.724109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8184-acb2-4c66-bc33-88c99a243e57 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.728434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 474/952] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:58.730795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.762497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.763345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.763807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.764451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.765219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.766777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:58.769712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING keystone.server.flask.common [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 05:16:58.795823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.sql.core [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:16:58.795823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 05:16:58.795823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [parameters: {'id': 'b87f04c3d38b4d3cbd4c9b50fc491b9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 05:16:58.795823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87f04c3d38b4d3cbd4c9b50fc491b9e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87f04c3d38b4d3cbd4c9b50fc491b9e. Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:16:58.804360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 26 05:16:58.805349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application yield resource Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise self.value Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application session.commit() Sep 26 05:16:58.807254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:16:58.808692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:16:58.809806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application result.read() Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:16:58.812519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [parameters: {'id': 'b87f04c3d38b4d3cbd4c9b50fc491b9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.813390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:58.814356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87f04c3d38b4d3cbd4c9b50fc491b9e. Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:58.815302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 479/953] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Sep 26 05:16:58 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:58.816830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.818926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.862905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c62f1-71a5-4ccf-bf38-657f019f6afc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.905493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.906143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.906479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.906759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.916290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.917838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:16:58.934032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:58.935059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:16:58.936466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:58.940371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 475/954] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:16:58.941402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850fb35-be1c-4853-9531-7162f6517e94 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:58.941635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:58.949599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:58.981950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:58.983396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:58.983798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:58.984826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:58.991231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:58.993236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-21f31e86-12bd-415a-b0a0-7e4f0e7b8d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:16:59.009002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:59.010175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:59.011113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 480/955] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:59.011829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20fd1-d9bd-4a0b-bca6-2a254082aac6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.012116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20fd1-d9bd-4a0b-bca6-2a254082aac6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.012418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20fd1-d9bd-4a0b-bca6-2a254082aac6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.017176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 481/956] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:59.017984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20fd1-d9bd-4a0b-bca6-2a254082aac6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.069596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81bde8-0891-42e3-8726-f88fc73c7d66 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.105080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.118160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.119135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.119530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.119902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.120757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:59.121159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.122749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30c87e-1529-4da0-8e5a-de5a4e0322ee None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.134185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20fd1-d9bd-4a0b-bca6-2a254082aac6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.145682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 476/957] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 05:16:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:59.145682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7ac9-ed5d-4269-a033-5ea39799f145 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.145682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7ac9-ed5d-4269-a033-5ea39799f145 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.145682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7ac9-ed5d-4269-a033-5ea39799f145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.162952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-81cd7ac9-ed5d-4269-a033-5ea39799f145 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:59.164658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-81cd7ac9-ed5d-4269-a033-5ea39799f145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:59.177064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.177799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.178175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.178547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.179311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:59.179710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.181296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.199971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 482/958] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:59.201552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.233316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 477/959] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d0f26-7932-4289-8594-3f97e30ebd63 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d0f26-7932-4289-8594-3f97e30ebd63 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d0f26-7932-4289-8594-3f97e30ebd63 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 478/960] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:16:59.252091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.252657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.252657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.278537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:16:59.280908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:16:59.329600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f036-d8ad-4499-99d2-247e6b12e332 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.379713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 479/961] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:16:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:16:59.383286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.383905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.384186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.384492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.456961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.458414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.486836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.536193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.571725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 483/962] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:16:59.576953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.591349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.649758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b26a0-77c2-4640-abda-25d0da35c308 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.688049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.688719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.688981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.689251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.690159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.691526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.692806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0377-67c7-4c1c-ad3c-f1bb32e2e76d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:59.700555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:59.701811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.702666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 480/963] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:59.703472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.728863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.739767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.740418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.740695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.741012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.751711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.753186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:16:59.767834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:59.768993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:16:59.770333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:16:59.770333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:16:59.770333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:16:59.770333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 484/964] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:59.770333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.818790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1370-95de-44dc-828c-b39a9b7db238 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.829174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:16:59.857566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.858170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.858404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.858673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.859327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.860637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:16:59.871087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:16:59.872086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:16:59.873017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 481/965] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:16:59.874402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:16:59.886278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33be279-1e76-4886-9012-02668b6d556f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:16:59.922644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:16:59.923401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:16:59.923681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:16:59.923951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:16:59.924607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:16:59.924910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:16:59.926311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:16:59.935801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 485/966] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:16:59.935801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.001015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:00.016219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8125791-a8bf-4c5f-a71e-354143b99b58 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.055179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.055796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.056075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.056377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.057164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.057811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.059473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.066287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f91c6-363d-45ca-9bde-5fe06c55b6ad None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.077835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 482/967] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:00.077835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.109314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.110466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.110798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.111065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.111690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.111988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.113385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.115528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:00.118431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 486/968] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:16:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:00.124222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.194404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688547a5-7cbf-4be5-b730-07ee4c938642 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.230708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88c9c-8c44-4f3a-99be-76ec35f1498b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.232383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.233088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.233450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.233791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.234447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.234751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.236186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.239462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:00.246933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 483/969] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:17:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:00.246933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.286967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.287964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.288470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.288941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.300170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.302799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:17:00.322842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:00.323949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:00.325015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 487/970] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:00.326609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.351325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:00.358430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38c370-18e6-49b9-a994-085f61141102 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.394834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.395365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.395642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.395913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.396687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.396985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.398327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.412285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 484/971] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:17:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:00.412285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.483003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8935-1a23-4dd8-ae98-7eb65efeb526 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.523261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.524062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.524445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.524779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.525518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.527187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.553250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c133ed-5e0f-43f2-91c1-d075e480f707 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.558290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 488/972] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:17:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:00.606444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.607597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.607868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.608132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.608853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.609154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.610582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45de76-8fbe-47cc-b286-d4fb4832d289 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.625785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 485/973] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:00.632330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:00.663697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:00.754744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5db36-a895-4fb8-af62-9129263ff212 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:00.789425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:00.790033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:00.790310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:00.790577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:00.791293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] RBAC: Adding query filter params (role.id=0046b1387736483ea5f7ccb93610540d, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=85e1f5c17df9495182082a9d85fa3401) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:00.791597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:00.793081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86f02f-1f6b-4eaa-aea4-12dcbf9a29fd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:00.799948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 489/974] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:00 2022] GET /identity/v3/role_assignments?user.id=85e1f5c17df9495182082a9d85fa3401&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=0046b1387736483ea5f7ccb93610540d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:01.613473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd73541-ae2f-4648-a8d6-0aa06b294aa9 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:01.614005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd73541-ae2f-4648-a8d6-0aa06b294aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:01.614238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd73541-ae2f-4648-a8d6-0aa06b294aa9 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:01.615903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 486/975] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:01.621660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:01.622064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:01.622428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:01.641877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:01.643531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:01.712620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 490/976] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:01.981515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd73541-ae2f-4648-a8d6-0aa06b294aa9 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:02.003383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:02.102008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd73541-ae2f-4648-a8d6-0aa06b294aa9 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:02.137779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:02.139039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.139605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.139897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.140569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:02.141986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:02.144612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING keystone.server.flask.common [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 05:17:02.157857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.sql.core [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:17:02.157857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 05:17:02.157857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [parameters: {'user_id': 'ea894b33372c4cd197b1f06ce3394b46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 05:17:02.157857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81543) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application result.read() Sep 26 05:17:02.175804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:17:02.176740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application yield resource Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise self.value Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application session.commit() Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:17:02.178130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:17:02.179045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:17:02.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application result.read() Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:17:02.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ea894b33372c4cd197b1f06ce3394b46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:02.181790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 26 05:17:02.183828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:02.186715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 487/977] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 05:17:01 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:02.196012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:02.212816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:02.306062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3258-11b1-4e24-8b09-024e03dc18cb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:02.345182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:02.345826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.346104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.346378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.353914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:02.355413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-be065204-5762-439c-8a08-b170fd3e2d83 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:17:02.367501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:02.368731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:02.370099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 491/978] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:02.377105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:02.461176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:02.484709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbb060-73f3-47df-821b-25ef74bed2c4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:02.521873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:02.522495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.522775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.523048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.523748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:02.524051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:02.525706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a9263-f918-445f-86da-c522c1d8aeb0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:02.567062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 488/979] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 05:17:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:02.705495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7145f-04e1-4f53-995e-8db3591584b9 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.705885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7145f-04e1-4f53-995e-8db3591584b9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.706079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7145f-04e1-4f53-995e-8db3591584b9 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.707655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 492/980] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:02.713374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cf3bf-23dc-4ec0-83b4-5e7002e9ae39 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.713903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cf3bf-23dc-4ec0-83b4-5e7002e9ae39 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.713903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cf3bf-23dc-4ec0-83b4-5e7002e9ae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.734278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fd2cf3bf-23dc-4ec0-83b4-5e7002e9ae39 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:02.735827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fd2cf3bf-23dc-4ec0-83b4-5e7002e9ae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:02.809822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 489/981] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:02.994274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49de9-f777-4666-8e78-4134c59a2da2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:02.994908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49de9-f777-4666-8e78-4134c59a2da2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:02.995380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49de9-f777-4666-8e78-4134c59a2da2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:02.997232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 493/982] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:03.002396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.002682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.002954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.022412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:03.023598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:03.039994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527138b-c558-451c-9455-29cd6d6503f9 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.040304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527138b-c558-451c-9455-29cd6d6503f9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.040577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527138b-c558-451c-9455-29cd6d6503f9 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.042160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 494/983] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:03.047189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.047475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.047749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.067748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:03.068957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:03.090977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.104295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 490/984] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:03.104295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.150225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 495/985] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:03.204817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd67d6-b740-4971-aa00-ed8935e7d2a3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.240703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.241299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.241574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.241839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.242475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.244071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.260020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 491/986] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:17:03 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:03.374062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.383208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.433424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.489674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443b6c-f80c-47ba-a5e6-b2019a9f4230 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.499183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.534923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.535468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.535745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.536019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.536641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.537992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:03.547494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:03.548673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:03.549530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 496/987] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:03.554400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.556439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8c8ea-d804-474b-a7b6-5de1f147f915 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.591107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.592395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.592682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.592941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] PATH_INFO: `/v3/services/image` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.593523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.594885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:03.604192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:03.605345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:03.606294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 492/988] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:03.612434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.629603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.669055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec602-a3ea-42f0-99ad-dabbce2cf1a2 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.706723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.707632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.707920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.708194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.708898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:03.709167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.710490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.720224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 497/989] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:03.721319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.724602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d22f-a1c4-429f-892c-05b68bd38d6c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.750384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.766919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.767891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.768173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.768478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.769434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] RBAC: Adding query filter params (name=image) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:03.770022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.771385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.777040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 493/990] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:03.782571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.831912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985b6c1-ce68-48d9-81cb-12b8e8eb9272 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.861834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:03.869352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e290c-f920-4e73-b057-3f012771203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.869960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e290c-f920-4e73-b057-3f012771203a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.870233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e290c-f920-4e73-b057-3f012771203a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.870496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e290c-f920-4e73-b057-3f012771203a None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.871111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e290c-f920-4e73-b057-3f012771203a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:03.871402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e290c-f920-4e73-b057-3f012771203a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.873339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e290c-f920-4e73-b057-3f012771203a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.878508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 498/991] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:03.884598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e290c-f920-4e73-b057-3f012771203a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:03.893449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9469-805e-4f47-89eb-533bd5707e45 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:03.930622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:03.931234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:03.931510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:03.931781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:03.932445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] RBAC: Adding query filter params (type=image) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:03.932727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:03.934073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:03.939720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 494/992] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:03.996660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e290c-f920-4e73-b057-3f012771203a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:04.034111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:04.034742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:04.035015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:04.035280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:04.036010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] RBAC: Adding query filter params (interface=admin, service_id=f019fce1560c40e688377a209fcc68a2, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:04.036345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:04.037711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01101009-3b19-46dd-932a-fbc295f8b7bd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:04.043611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 499/993] 10.222.0.49 () {56 vars in 1249 bytes} [Mon Sep 26 05:17:03 2022] GET /identity/v3/endpoints?service_id=f019fce1560c40e688377a209fcc68a2&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:04.050141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:04.112689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:04.167027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3c0bf-61b7-446b-9871-f5f52de3852a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:04.203650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:04.204278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:04.204570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:04.204846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:04.205458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:04.206816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db83ef-68e8-4c94-a929-24c4fe0767eb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:04.220336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 495/994] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:17:04 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:17:05.613596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871071e9-fa68-41c2-a0f6-72ba2d0799ea None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:05.614135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871071e9-fa68-41c2-a0f6-72ba2d0799ea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:05.614379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871071e9-fa68-41c2-a0f6-72ba2d0799ea None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:05.616304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 500/995] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:05.623107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:05.623527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:05.623669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:05.642549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:05.643808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:05.716616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 496/996] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:05.988409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871071e9-fa68-41c2-a0f6-72ba2d0799ea None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.051889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.104055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871071e9-fa68-41c2-a0f6-72ba2d0799ea None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.142615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.143229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.143501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.143766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.144363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.145711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e41c6e6f-ad75-48d1-81bc-9257d13ae467 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:06.155233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:06.156238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:06.157199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:06.160586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 501/997] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:06.162164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.208797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117db2-3656-4713-9c5d-52a8b2e7472e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.209141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117db2-3656-4713-9c5d-52a8b2e7472e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.209367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117db2-3656-4713-9c5d-52a8b2e7472e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.209660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.210715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 502/998] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:06.216918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.217215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.217533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.235893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:06.237099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:06.274037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92e40d-b5e9-4167-89c9-2870de410bf4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.313200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 503/999] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:06.320051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.320697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.320971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.321235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.321867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:06.322162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.323509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b247-30d0-43d6-aad0-f24f6df58f5a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.328994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 497/1000] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:06.334372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.388662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.409436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7367e-6f67-4936-9ef2-abfaeb0409e8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.409756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7367e-6f67-4936-9ef2-abfaeb0409e8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.409973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7367e-6f67-4936-9ef2-abfaeb0409e8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.412281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 498/1001] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:06.417229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.417514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.417787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.436428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:06.437602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:06.446134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb5e5e-8625-4805-bb52-e0f618a527e3 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.486514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.487122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.487413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.487678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.488319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:06.488620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.489981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.495317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 504/1002] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:06.501021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.509931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 499/1003] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:06.584173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.604862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db80865-0011-4902-9ab3-9cbc60907e41 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.623630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.627592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.646190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.646805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.647078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.647684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.648029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] RBAC: Adding query filter params (interface=public, service_id=889a8955c5174363b085c49423ca8e07, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:06.648368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.649752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.661073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 505/1004] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/endpoints?service_id=889a8955c5174363b085c49423ca8e07&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:06.662124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.703873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c1400-325c-4879-9b2e-e6c2e891d485 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.740334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.741183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.741461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.741732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.742384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.744030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.762438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 500/1005] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:17:06 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:17:06.775478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a419069-f089-4b77-b1ed-d4d51be63e4b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.781496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:06.823726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.824561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.824941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.825315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.826351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.827870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.835350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 506/1006] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:17:06.893232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:06.903074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817240-524c-4e32-9941-ef1ec4c25e2f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:06.944186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:06.944954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:06.945299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:06.945640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:06.946302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:06.947751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:06.960062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:06.961389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:06.962347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 501/1007] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:06.969121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:07.020844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:07.087172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df9a46-9b28-45c2-8ce5-1c2ffc65c8bf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:07.128422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:07.129280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:07.129719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:07.130136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:07.130898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:07.131342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:07.132881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:07.139156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 507/1008] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:17:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:07.145000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:07.250113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6c89c-c798-44b4-beaa-b272d8507c0f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:07.287458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:07.288069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:07.288379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:07.288651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:07.289353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:07.289660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:07.291030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e9b7b-aafb-4e0f-a06a-8100a451fb75 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:07.296558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 502/1009] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 05:17:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:07.302059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:07.407619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc7242-a8ce-4cdd-acc0-522b06b0f5a6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:07.409966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:07.444732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:07.445341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:07.445615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:07.445882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:07.446561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:07.447919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3635a86-01c6-4c36-9e13-a8fedbf20ea5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:07.472279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 508/1010] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:17:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:09.032453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b70b5-b9c9-4c2c-a00b-e71b1057dcf3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.032986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b70b5-b9c9-4c2c-a00b-e71b1057dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.032986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b70b5-b9c9-4c2c-a00b-e71b1057dcf3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.036287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 503/1011] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:09.039582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.040013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.040187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.060285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:09.061587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:09.075194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e078c1-6227-49db-868e-ed632f109792 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.076133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e078c1-6227-49db-868e-ed632f109792 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.076654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e078c1-6227-49db-868e-ed632f109792 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.081620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 504/1012] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:09.088809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.089341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.090439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.119463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:09.120713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:09.142099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:09.154513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 509/1013] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:09.195420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 505/1014] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:09.426190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.466586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.525551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:09.543090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af132a-e05e-4c1a-ba8e-7ee9ab869ef0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:09.574255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eebd44-0021-49f7-98b6-777936aa80f0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:09.595682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:09.596795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.597072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.597470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.598194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:09.600036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:09.610060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:09.610259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:09.611205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:17:09.612082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:09.612868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.612868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.612868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] PATH_INFO: `/v3/services/image` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.612868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:09.613785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:09.616332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 510/1015] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:09.617220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:09.623509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:09.624584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:09.629381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 506/1016] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:09.630451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.652010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:09.724434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba0c7e-0dc6-4f21-a504-d8fbdd8234bb None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:09.737114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bc09-f300-46c9-a34a-c7ee9bb67b77 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:09.757670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:09.758277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.758549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.758813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.759423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:09.759715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:09.761098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471d5f2-e1de-46c5-82ba-6249a0fa2d73 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:09.766488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 511/1017] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:09.767657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97d2a-763e-48b3-8076-dc83159c9754 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.767936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97d2a-763e-48b3-8076-dc83159c9754 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.768203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97d2a-763e-48b3-8076-dc83159c9754 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.769440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 512/1018] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:09.775508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.775791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.776067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.778280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:09.779787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.780209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.780518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.781168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] RBAC: Adding query filter params (name=image) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:09.781458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:09.782817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:09.787940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 507/1019] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:09.788789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.795784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:09.797116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:09.811187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:09.866204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 513/1020] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:09.867057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.873669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:09.894072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6d17d-4e62-403f-8326-54fc71819603 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:09.930214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:09.930817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:09.931098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:09.931371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:09.931991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:09.932317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:09.933678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:09.943943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 508/1021] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:09.944944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:09.978932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460688c-52d5-4ea5-a82d-8c5566ef782e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.014120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.014755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.015027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.015291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.015911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] RBAC: Adding query filter params (type=image) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:10.016204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.017587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.022476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 514/1022] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:10.028140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:10.042572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:10.073748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33187-07b2-49d0-82c6-fedffda039cc None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.117129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.117743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.118049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.118527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.119196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.120587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.138526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58e32-9802-4674-bcae-cf3bc34ea22f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.142606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 509/1023] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:17:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:10.143405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:10.152993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:10.185290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.186096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.186329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.186562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.187265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] RBAC: Adding query filter params (interface=public, service_id=a29be0d8a83242a99b06c590c0249a47, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:10.187775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.189183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.199973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 515/1024] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:17:10 2022] GET /identity/v3/endpoints?service_id=a29be0d8a83242a99b06c590c0249a47&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:10.202104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:10.275136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2a831-cb31-477a-a6b3-728def444e23 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.278536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:10.312471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.313139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.313413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.313677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.314818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.316088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeaa5-af7b-4ba3-ba07-fac65d81a33d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.316268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.317876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 510/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:10.323934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:10.359822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.359822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.360138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.360341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.360846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.362209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7895f6-2183-4482-b51e-86b91972c0a0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.369516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 516/1026] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:17:10 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:17:10.412591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:10.453398 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf333a-7d7c-4e14-980a-245acde1feea None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:10.491421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:10.492048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:10.492352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:10.492625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:10.493541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:10.495001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d4478-5193-486d-aaa5-968b7b118061 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:10.526442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 511/1027] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:17:10 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:17:12.443489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fd0d-a2c4-4e3b-93fd-244ef6794c72 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.444023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fd0d-a2c4-4e3b-93fd-244ef6794c72 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.444023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016fd0d-a2c4-4e3b-93fd-244ef6794c72 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.445433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 517/1028] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:12.450541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4845cef-ba8a-4b00-b5c3-abb41ee7736b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.450961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4845cef-ba8a-4b00-b5c3-abb41ee7736b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.451207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4845cef-ba8a-4b00-b5c3-abb41ee7736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.469927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b4845cef-ba8a-4b00-b5c3-abb41ee7736b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:12.471111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b4845cef-ba8a-4b00-b5c3-abb41ee7736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:12.548183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 512/1029] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:12.810076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e694-8585-48a6-9e43-a2b1c9cbc3e1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.810280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcec54-d07d-42ea-b708-d7d63ac626e8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.810368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e694-8585-48a6-9e43-a2b1c9cbc3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.810659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcec54-d07d-42ea-b708-d7d63ac626e8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.810773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e694-8585-48a6-9e43-a2b1c9cbc3e1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.810849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcec54-d07d-42ea-b708-d7d63ac626e8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.812171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 518/1030] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:12.812369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 513/1031] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:12.816964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.817151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:12.817248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.817409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:12.817512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.817656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:12.835888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:12.836077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:12.837085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:12.837205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:12.884644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:12.906138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 514/1032] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:12.906839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:12.912388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 519/1033] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:13.015593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5be2b-df1f-4c7c-9249-8f9a8f4a141d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.050132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.050766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.051042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.051309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.062383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:13.067096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:13.068070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:13.073360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 515/1034] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:17:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:13.074727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.148241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.181663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.184222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca9873-f09b-4902-b75e-215b519d05ce None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.220879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.221481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.221756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.222024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.222675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:13.222974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.224369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.228202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.233360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 520/1035] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:13.234150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.289686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb9374-ea77-409e-a4c4-8df92b46b472 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.324473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.325021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.325912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.326182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.327283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.328745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.330302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 516/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:13.331055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.332519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f618-4d5b-430b-a3fb-f9dd796fecaf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.359111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.367455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.368087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.368666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.368937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] PATH_INFO: `/v3/services/image` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.369508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.370866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:13.380309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:13.381549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:13.382604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 521/1037] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:13.383615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.435057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855da7c-54cc-4144-b438-e188ed02f0d9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.446349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.469742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.470367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.470631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.470899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.481118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.483039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.484651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5488d-e4fa-4a40-b932-8155b3a6391d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:13.501377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:13.502516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:13.503432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:13.503432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:13.503432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:13.503432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 517/1038] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:13.503432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.521077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.521686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.521960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.522232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.522874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.524316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.546023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 522/1039] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:17:13 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:13.546834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.600346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.630122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751efff-0166-47f8-b61f-6f76ff80b342 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.662807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b66b4-655b-45b5-aabe-1d9f9df92b61 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.671278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.671912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.672184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.672488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.673134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] RBAC: Adding query filter params (name=image) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:13.673439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.674860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.680622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 518/1040] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:13.687282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.704668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.705289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.705571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.705837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.706486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:13.706891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.708440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.720226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.722158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 523/1041] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:13.727784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.811366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4da17-eca0-4337-a010-6ec3386fbd22 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.836177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538d18-7025-4f7c-81c9-5b1836622a1a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:13.846962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.848785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.849407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.849698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.850005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.850812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] RBAC: Adding query filter params (type=image) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:13.851085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.852590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.858538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 519/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:13.864078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.879382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:13.880020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:13.880326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:13.880597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:13.888224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:13.889795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:17:13.902395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:13.904393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:13.905749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 524/1043] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:13.911466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:13.970892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:13.984230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c41d5-a319-4c22-9994-b6dc42085f28 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:14.025178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:14.025787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:14.026058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:14.026324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:14.026982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:14.027223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643fad4-e311-4c24-9907-e83647735a63 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:14.029255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:14.051624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 520/1044] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:17:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:17:14.069834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:14.070467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:14.070743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:14.071012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:14.071749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:14.072051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:14.073571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:14.104199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 525/1045] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:17:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:14.108065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:14.237723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90303b89-50ed-4603-90f7-e146b1e85cef None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:14.274693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:14.275313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:14.275599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:14.275870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:14.292485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:14.364766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:14.367424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:14.509752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 521/1046] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:14.510885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:14.536571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:14.620906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1103-3469-42cb-9953-578a8c207ddf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:14.657505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:14.658203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:14.658480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:14.658750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:14.670103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:14.671578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:14.687246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:14.688297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:14.693159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:14.693159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:14.693159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:14.693159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 526/1047] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:14.695899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:14.747477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:14.823942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f527-0774-4a57-bca6-32a70eb85af1 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:14.859130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:14.859737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:14.860013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:14.860310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:14.860946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:14.861245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:14.862592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:14.879641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 522/1048] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:14.880936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:14.960573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:14.988726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84853650-b5f8-4ced-af57-98f405496fe0 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:15.039300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:15.039928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:15.040204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:15.040497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:15.047077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:15.048541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:17:15.059540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:15.060476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:15.068299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 527/1049] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:15.068943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:15.178716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212e2a-b787-4f80-bb40-010658c896e2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:15.187060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:15.215441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:15.216446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:15.216656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:15.216963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:15.217578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:15.217871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:15.219296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83803dd1-fd1c-45b6-8d83-145325e89bc2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:15.236714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 523/1050] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:15.238019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:15.344054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790577f-bf47-45a1-b5f0-2fdd1d00d2e5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:15.378524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:15.379130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:15.379405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:15.379674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:15.380418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] RBAC: Adding query filter params (role.id=0046b1387736483ea5f7ccb93610540d, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:15.380726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:15.382149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea919a5-7e5e-41ea-91cc-ddccb20809a2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:15.383922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:15.389878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 528/1051] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:15 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=0046b1387736483ea5f7ccb93610540d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:15.785077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1dba-1b34-424a-908a-183a6352616a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:15.785624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1dba-1b34-424a-908a-183a6352616a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:15.785624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1dba-1b34-424a-908a-183a6352616a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:15.786936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 524/1052] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:15.792136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911f2fc-3d84-46c1-b260-9ad775ec5361 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:15.792578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911f2fc-3d84-46c1-b260-9ad775ec5361 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:15.792759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911f2fc-3d84-46c1-b260-9ad775ec5361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:15.812509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3911f2fc-3d84-46c1-b260-9ad775ec5361 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:15.813676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3911f2fc-3d84-46c1-b260-9ad775ec5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:15.882783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 529/1053] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:16.146769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb1dba-1b34-424a-908a-183a6352616a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:16.256598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb1dba-1b34-424a-908a-183a6352616a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:16.294796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:16.295419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.295692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.296064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.306274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:16.311230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:16.312172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:16.313418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:16.315850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b43d-ac85-4b99-a970-58efa5b6a571 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.316143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b43d-ac85-4b99-a970-58efa5b6a571 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.316604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b43d-ac85-4b99-a970-58efa5b6a571 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.316729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 525/1054] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:16.318014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 530/1055] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:16.318118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:16.322813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.323091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.323364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.342364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:16.343539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:16.347759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:16.418201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 531/1056] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:16.429096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88d37-99af-43c0-9fed-4bf355935f54 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:16.464624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:16.465226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.466568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.466853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.467579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:16.467880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:16.469328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:16.478496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 526/1057] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:16.484293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:16.543126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:16.585080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb799-8ee9-4283-b6ba-13a70f933d67 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:16.619360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:16.619987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.620278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.620550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] PATH_INFO: `/v3/users/2fcbdbd52c904b35b2df20ed84747897` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.634225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcbdbd52c904b35b2df20ed84747897)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:16.635778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:16.638213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 532/1058] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/users/2fcbdbd52c904b35b2df20ed84747897 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:16.643674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:16.691700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:16.749217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:16.762201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0e5e2-15b0-427b-b31a-19138d581699 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:16.800456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:16.801122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.801436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.801604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c87619-cdd5-4711-a4a0-4cb9da6795d5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:16.802001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.820966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:16.841406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:16.842008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:16.842280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:16.842548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:16.875268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:16.876748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:16.915180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:16.916672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:16.969120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:16.982776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 527/1059] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:17:16.988565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:17.036297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 533/1060] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:17:17.038256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:17.056070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:17.101154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479dc9f-604d-43fd-bd67-e0d9b64db461 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:17.137915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:17.138721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.138998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.139264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.140499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:17.141840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:17.143453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 528/1061] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:16 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:17.149245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:17.154195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de4cc-669b-4a01-8dd3-8fd9a2a84b63 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:17.189220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:17.189853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.190125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.190397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.201219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:17.202650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:17.210382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:17.218135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:17.219110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:17.220053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:17.220053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:17.220053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:17.220053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 534/1062] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:17.225015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:17.264839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bc28-dd6c-4187-8627-2767d7ae020b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:17.292666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:17.303146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:17.303863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.304139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.304432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.305146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=536062968e0944c4b04b98b83db365da, user.id=2fcbdbd52c904b35b2df20ed84747897) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:17.305449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:17.307256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84711-97bd-49d6-af75-c11449a92059 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:17.313718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 529/1063] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:17 2022] GET /identity/v3/role_assignments?user.id=2fcbdbd52c904b35b2df20ed84747897&scope.project.id=536062968e0944c4b04b98b83db365da&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:17.330817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995ba1-dff2-40af-829a-b388b51f3845 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:17.368001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:17.368914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.369194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.369465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.370124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:17.370424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:17.372592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355baf73-8acb-432d-906c-7a16663ddf8b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:17.392309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 535/1064] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:17.577752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332231-1796-48e3-901c-e1940459db4f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.577967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332231-1796-48e3-901c-e1940459db4f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.578245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2332231-1796-48e3-901c-e1940459db4f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.579843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 530/1065] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:17.584938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:17.585227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:17.585503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:17.604141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:17.605361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:17.678834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 536/1066] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:17.946470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2332231-1796-48e3-901c-e1940459db4f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.047848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2332231-1796-48e3-901c-e1940459db4f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.067644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:18.084846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.085474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.085744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.086006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.097110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:18.101881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:18.104397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:18.105288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 531/1067] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:17:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:18.109400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.168781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:18.215350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce630abb-48ad-4e8f-ac12-7c2bdccda9e4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.250188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.250805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.251085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.251358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.252027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:18.252365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:18.253747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:18.262297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 537/1068] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:18.268912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.372541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a55286-0c3e-4043-8b1f-b805df166510 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.406619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.407238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.407512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.407779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.417593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:18.419026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:18.433017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:18.433951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:18.435575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:18.435575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:18.435575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:18.436629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 532/1069] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:18.440276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.537481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b03a8-ec61-4a93-b63a-7f1e46b29d55 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.556226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:18.570642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.571248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.571527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.571798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.572452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:18.572751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:18.574103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:18.586308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 538/1070] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:18.591919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.623978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:18.690600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6949d-b391-462b-97d5-27217a585195 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.722702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.723318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.723599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.723868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.731763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:18.733231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:17:18.744449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:18.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:18.746458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 533/1071] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:18.751900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:18.847175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3396d-db89-47e7-98c4-295d5fdd41b7 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:18.883597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:18.884188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:18.884497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:18.884768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:18.885481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:18.885783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:18.887243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:18.903764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 539/1072] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:18.909427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:19.000421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:19.009306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f3861-3155-4240-9577-46383cb5d033 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:19.046396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:19.047026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.047304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.047576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.058261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:19.059790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:19.073788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:19.075025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:19.075979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:19.075979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:19.075979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:19.075979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 534/1073] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:19.080856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:19.125205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:19.185221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780ae3e-51ed-4c3b-825b-b74b81cc78b4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:19.218962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:19.219562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.219834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.220101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.220752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:19.221049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:19.222393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:19.234194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 540/1074] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:19.239753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:19.337119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132962f-9eed-4854-9893-d5d25b24eb31 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:19.370781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:19.371319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.371678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.371994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.378675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:19.380128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:17:19.392513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:19.393620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:19.394536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 535/1075] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:19.399393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:19.498924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:19.502611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51bf0-b5bb-4327-be02-31bd412e92b3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:19.537354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:19.537955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.538229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.538495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.539189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:19.539492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:19.540954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886fccd-c463-4b12-8e34-511a7a9dbe97 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:19.553364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 541/1076] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:19.559340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:19.563180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:19.586444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171ef8-aaff-46fa-9956-1cec70962e5e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.586723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171ef8-aaff-46fa-9956-1cec70962e5e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.586996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171ef8-aaff-46fa-9956-1cec70962e5e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.588353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 542/1077] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:19.593856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.594132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.594401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.611893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:19.613208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:19.670060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab0070-3224-4fbd-bec1-ad23db646f3f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:19.687108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 543/1078] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:19.705157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:19.705812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:19.706088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:19.706354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/0046b1387736483ea5f7ccb93610540d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:19.708471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=2eadef8d5ac94a3eb3da8d0d3a66d7b5, role_id=0046b1387736483ea5f7ccb93610540d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:19.709882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:19.723712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 536/1079] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:17:19 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/2eadef8d5ac94a3eb3da8d0d3a66d7b5/roles/0046b1387736483ea5f7ccb93610540d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:17:19.989217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:20.078526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:20.094093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58169028-f6b7-4d39-9857-7e16a5aa3952 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:20.131913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:20.132616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:20.132851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:20.133124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:20.143794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:20.148639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:20.149778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:20.150681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 544/1080] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:20.156213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:20.218103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:20.257494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276355a0-f923-4f78-a259-b25cac49f426 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:20.289089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:20.289702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:20.289978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:20.290246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:20.290904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:20.291203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:20.292588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:20.302366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 537/1081] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:20.308368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:20.409668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46184bf-44f1-4be8-810f-e28897baebb9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:20.444091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:20.444744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:20.445017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:20.445287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] PATH_INFO: `/v3/users/2fcbdbd52c904b35b2df20ed84747897` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:20.446401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcbdbd52c904b35b2df20ed84747897)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:20.447824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:20.449432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 545/1082] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:20 2022] GET /identity/v3/users/2fcbdbd52c904b35b2df20ed84747897 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:20.455573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:20.552657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176070f1-65c7-4b66-afdd-414bc8ee808a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:20.577832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:20.586773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:20.587409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:20.587689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:20.587959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:20.589116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:20.591084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494a3ad-29d1-4cc0-ad1c-b51955ff18bb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:20.592759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 538/1083] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:20 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:20.599012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:20.621714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:20.703795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949469f-45bf-476a-88a1-3aa3b9f22523 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:20.737609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:20.738257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:20.738537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:20.738811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da/users/2fcbdbd52c904b35b2df20ed84747897/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:20.740737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] RBAC: Authorizing `identity:create_grant(project_id=536062968e0944c4b04b98b83db365da, user_id=2fcbdbd52c904b35b2df20ed84747897, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:20.742169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3faa-8f1c-4143-81da-ad152bdb8fce None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:20.755321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 546/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:17:20 2022] PUT /identity/v3/projects/536062968e0944c4b04b98b83db365da/users/2fcbdbd52c904b35b2df20ed84747897/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:17:22.018447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bebe9-c764-4830-8254-2a9e8f19007a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.019048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bebe9-c764-4830-8254-2a9e8f19007a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.019048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bebe9-c764-4830-8254-2a9e8f19007a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.020355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 539/1085] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:22.025905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871166-8a84-4389-9136-20fc77682329 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.026370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871166-8a84-4389-9136-20fc77682329 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.026470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871166-8a84-4389-9136-20fc77682329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.045719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ad871166-8a84-4389-9136-20fc77682329 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:22.046891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ad871166-8a84-4389-9136-20fc77682329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:22.115416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 547/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:22.393591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bebe9-c764-4830-8254-2a9e8f19007a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:22.517857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bebe9-c764-4830-8254-2a9e8f19007a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:22.551719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:22.552369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.552650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.552922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.563577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:22.568402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:22.569570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:22.570849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 540/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 05:17:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:22.575446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad871166-8a84-4389-9136-20fc77682329 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:22.583398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:22.675295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad871166-8a84-4389-9136-20fc77682329 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:22.709842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:22.710455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.710732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.711005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.711666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:22.711965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:22.713378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aaeb2-f85a-4673-8ecd-58683f8c31e5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:22.724330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 548/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:17:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:22.730023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:22.792870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:22.838257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dac5bb-6fc5-4999-8ad1-ea1d77e0a531 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:22.873272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:22.873900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.874181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.874452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.887034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:22.889518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e54c6-86f7-400f-98fc-cd7af876dd26 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.889791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:22.889899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e54c6-86f7-400f-98fc-cd7af876dd26 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.890061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e54c6-86f7-400f-98fc-cd7af876dd26 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.891398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 549/1089] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:17:22.896650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:22.896934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:22.897231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:22.915450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:22.915774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:22.916959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:22.917986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:22.917986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:22.917986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:22.918171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:22.919007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 541/1090] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:22.927212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:22.982617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:22.988550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 550/1091] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:23.031645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab40fc-bbea-4f26-bb8f-1482da024ef9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.039165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.067019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.067618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.067894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.068158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.068815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:23.069113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.070453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.084944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 542/1092] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:23.091300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.190532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810014-0e0c-42af-8fc1-4c190052e1aa None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.224793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.225443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.225721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.225992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.232887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.234320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:17:23.245858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:23.246916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:23.248311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 551/1093] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:23.252783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.305754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.355478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0231f71-7f07-4588-86e1-9cf9ca778bf6 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.364300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.389687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.390297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.390575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.390846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.391562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:23.391866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.393340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.406967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba6daa-d32d-432a-95fd-8c096ee2c113 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.410825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 543/1094] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:23.416586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.440697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.440873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.441354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.441630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.441907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.452770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:23.457958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:23.458976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:23.459896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 552/1095] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:23.464887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.515375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c06f4-3ff5-4b00-8e5d-dff27efe00d3 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.550100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.550760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.550975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.551250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.562454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf0b5c-5fc6-4c09-a0b7-e9a2f6c78c9d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.579482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.596848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.597447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.597719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.597985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.598629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:23.598924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.600334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.609078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 553/1096] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:23.614977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.618682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.620107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.643883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.712929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8edb-8349-42b5-a059-28c0923f10a9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.721059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 544/1097] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:17:23.726849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.751217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.751837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.752108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.752413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] PATH_INFO: `/v3/users/2fcbdbd52c904b35b2df20ed84747897` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.753527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcbdbd52c904b35b2df20ed84747897)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.754944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.756534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 554/1098] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/users/2fcbdbd52c904b35b2df20ed84747897 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:23.763039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.810821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.825716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8569-e108-467c-b8cd-a62206b3e8eb None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.849762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:23.860859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.861213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae33dcc-03aa-44c4-b024-d38e171d9134 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:23.861482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.861756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.862021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.872846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.874300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:23.889649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:23.890632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:17:23.891874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:23.891874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:17:23.891874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:23.891874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 545/1099] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:23.896429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:23.897106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:23.897716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:23.897990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:23.898259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:23.965829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:23.967748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:23.997813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d7c5-ca75-4777-92c0-7b29828423bd None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.014027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:24.032349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.032954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.033224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.033486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.034110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:24.034407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.035940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.047258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 546/1100] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:24.052833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:24.065328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 555/1101] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:17:24.070896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:24.094171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:24.161519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1e0e2-b83e-4a4d-b640-55434e728b01 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.170517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dab84-23d4-440e-a430-f57c92022184 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.195400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.196031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.196336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.196613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.203725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.204723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.205198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.205344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.205612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.205875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.206968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.208390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.210159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 556/1102] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:24 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:24.216370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:17:24.216707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:24.217745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:17:24.218898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:24.224535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 547/1103] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:17:24.229670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:24.236810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:24.317807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf03e-be1f-46de-892d-6a85c7a1697d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.331517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:24.337513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d5a-39ae-4054-8c0c-c903ed6170b3 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.352084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.352713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.352983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.353248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.353952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=536062968e0944c4b04b98b83db365da, user.id=2fcbdbd52c904b35b2df20ed84747897) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:24.354250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.355688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.362123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 557/1104] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:24 2022] GET /identity/v3/role_assignments?user.id=2fcbdbd52c904b35b2df20ed84747897&scope.project.id=536062968e0944c4b04b98b83db365da&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:17:24.374409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.375036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.375310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.375575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.376295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:24.376601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.378047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba415f11-4c05-4974-8375-4cc207beff88 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.392339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 548/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:24.398748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:24.484672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:24.504964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43f15-f9f0-4a25-a92a-c84d8b2e694d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:24.539402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:24.540000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:24.540294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:24.540563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:24.541270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] RBAC: Adding query filter params (role.id=0046b1387736483ea5f7ccb93610540d, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:24.541570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:24.542980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c154-edff-427b-ad25-a688072cb3c4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:24.550425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 558/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:24 2022] GET /identity/v3/role_assignments?user.id=2eadef8d5ac94a3eb3da8d0d3a66d7b5&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=0046b1387736483ea5f7ccb93610540d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:26.620421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e0032-bd5b-4298-8b9f-d7b7d0735705 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:26.620958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e0032-bd5b-4298-8b9f-d7b7d0735705 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:26.620958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e0032-bd5b-4298-8b9f-d7b7d0735705 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:26.622287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 549/1107] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:26.627522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:26.627939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:26.628058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:26.646166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:26.647333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:26.712470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 559/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:26.981197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e0032-bd5b-4298-8b9f-d7b7d0735705 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:27.080529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e0032-bd5b-4298-8b9f-d7b7d0735705 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:27.118735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:27.119358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:27.119636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:27.119906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:27.121048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:27.122934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6fafe-4ad0-48e5-9528-cdbf6c024223 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:27.124534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 550/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:27.130289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:27.207220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:27.237008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ce81f-0728-4e6e-9ea4-fabc8b2d5955 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:27.271482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:27.272084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:27.272395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:27.272671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:27.273326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:27.274754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e295d-253a-4d7b-911c-529b872b989f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:27.295364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 560/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:17:27 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:29.516814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80858277-1468-4eea-aae2-563569240ebc None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:29.517380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80858277-1468-4eea-aae2-563569240ebc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:29.517380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80858277-1468-4eea-aae2-563569240ebc None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:29.518665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 551/1111] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:29.524012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:29.524483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:29.524585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:29.542523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:29.544116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:29.609459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 561/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:29.880061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80858277-1468-4eea-aae2-563569240ebc None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:29.904315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:29.976435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80858277-1468-4eea-aae2-563569240ebc None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:30.010052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:30.010663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:30.010938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:30.011205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] PATH_INFO: `/v3/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:30.012661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] RBAC: Authorizing `identity:get_role(role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:30.014004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:30.015294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 552/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:29 2022] GET /identity/v3/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:30.021070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:30.079192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:30.122367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915127-f4b7-491b-92cf-40c6ed86f02d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:30.155755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:30.156341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:30.156620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:30.156916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] PATH_INFO: `/v3/users/89ea05e1ac114bb3809d7e677df024c6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:30.170267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea05e1ac114bb3809d7e677df024c6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:30.171714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:30.173339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 562/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:30 2022] GET /identity/v3/users/89ea05e1ac114bb3809d7e677df024c6 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:30.179339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:30.268212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:30.277983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f090-ec8c-418a-be22-47e9b2f7823b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:30.310974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:30.311578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:30.311855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:30.312126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:30.376928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:30.378346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:30.466737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:30.476381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 553/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:30.483163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:30.578612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146513e-16d1-47e4-a887-467fd2191125 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:30.596824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:30.613237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:30.613864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:30.614136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:30.614402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:30.615499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:30.616961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783736c-482a-44eb-aaad-2bb6da620733 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:30.618840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 563/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:30 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:30.625104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:30.724999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed97c-96f3-4dce-86c8-97f34ce2dede None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:30.758314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:30.758907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:30.759185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:30.759456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:30.760161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=536062968e0944c4b04b98b83db365da, user.id=89ea05e1ac114bb3809d7e677df024c6) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:30.760494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:30.762541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2b19d-1406-4b8f-973d-4ce13ffbdfe7 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:30.767667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 554/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:30 2022] GET /identity/v3/role_assignments?user.id=89ea05e1ac114bb3809d7e677df024c6&scope.project.id=536062968e0944c4b04b98b83db365da&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:32.875751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61f77-9af9-4963-a54b-70ad1820b53f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:32.876298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61f77-9af9-4963-a54b-70ad1820b53f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:32.876413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61f77-9af9-4963-a54b-70ad1820b53f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:32.877668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 564/1118] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:32.882918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:32.883321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:32.883470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:32.901211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:32.902431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:32.951160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:32.967197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 555/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:33.232560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff61f77-9af9-4963-a54b-70ad1820b53f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:33.330348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff61f77-9af9-4963-a54b-70ad1820b53f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:33.363251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:33.363866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:33.364144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:33.364439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] PATH_INFO: `/v3/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:33.365811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] RBAC: Authorizing `identity:get_role(role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:33.367171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:33.368767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 565/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:33 2022] GET /identity/v3/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:33.374631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:33.470395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bc86b-6a50-4298-8620-5cbe5ada0c31 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:33.506236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:33.506785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:33.507061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:33.507328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] PATH_INFO: `/v3/users/89ea05e1ac114bb3809d7e677df024c6` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:33.508461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea05e1ac114bb3809d7e677df024c6)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:33.509883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:33.511447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 556/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:33 2022] GET /identity/v3/users/89ea05e1ac114bb3809d7e677df024c6 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:33.517341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:33.570693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:33.614565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e7353-f8f0-483f-8fb7-f6e9ba80a301 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:33.647367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:33.647978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:33.648273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:33.648551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:33.649651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:33.651059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b799486-d305-44c5-a5bb-bd3eabf451e2 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:33.652891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 566/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:33 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:33.658739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:33.712269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:33.755849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcda0e-c8bd-49de-bd22-64202f1ed1a8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:33.789545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:33.790136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:33.790412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:33.790682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da/users/89ea05e1ac114bb3809d7e677df024c6/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:33.792833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] RBAC: Authorizing `identity:create_grant(project_id=536062968e0944c4b04b98b83db365da, user_id=89ea05e1ac114bb3809d7e677df024c6, role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:33.794231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943119f5-5e4f-4db6-b3bc-db7636e5077d None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:33.806217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 557/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:17:33 2022] PUT /identity/v3/projects/536062968e0944c4b04b98b83db365da/users/89ea05e1ac114bb3809d7e677df024c6/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:17:36.022710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c63c0-017c-43cf-be7f-47650cebdc6e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:36.023244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c63c0-017c-43cf-be7f-47650cebdc6e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:36.023244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c63c0-017c-43cf-be7f-47650cebdc6e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:36.024623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 567/1124] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:36.030281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:36.030693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:36.030830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:36.048978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:36.050141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:36.116931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 558/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:36.388033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c63c0-017c-43cf-be7f-47650cebdc6e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:36.485383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c63c0-017c-43cf-be7f-47650cebdc6e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:36.506977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:36.518133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:36.518747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:36.519020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:36.519284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] PATH_INFO: `/v3/roles/96685976a13e4180ba58f74d6dc0cb4f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:36.520651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] RBAC: Authorizing `identity:get_role(role_id=96685976a13e4180ba58f74d6dc0cb4f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:36.522008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:36.523297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 568/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:36 2022] GET /identity/v3/roles/96685976a13e4180ba58f74d6dc0cb4f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:36.529717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:36.573316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:36.628017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72380-9510-449c-80fa-bf00b78e3fac None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:36.660241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:36.660831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:36.661115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:36.661391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] PATH_INFO: `/v3/users/89ea05e1ac114bb3809d7e677df024c6` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:36.662494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea05e1ac114bb3809d7e677df024c6)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:36.663913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:36.665505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 559/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:36 2022] GET /identity/v3/users/89ea05e1ac114bb3809d7e677df024c6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:36.671855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:36.767131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf4c0-7862-4f2d-b7c2-720a17eb2a0b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:36.799721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:36.800335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:36.800610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:36.800875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:36.854287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:36.865058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:36.866497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:36.961961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 569/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:36.968498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:37.066211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60b94-2497-49c1-8155-c1e34c5c4bb9 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:37.078990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:37.100643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:37.101268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:37.101546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:37.101816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] PATH_INFO: `/v3/projects/536062968e0944c4b04b98b83db365da` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:37.102925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] RBAC: Authorizing `identity:get_project(project_id=536062968e0944c4b04b98b83db365da)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:37.104373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711b0b4-f600-47ff-af45-9a3e2bd0851c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:37.106870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 560/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:36 2022] GET /identity/v3/projects/536062968e0944c4b04b98b83db365da => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:37.113029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:37.199027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:37.208872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd64-780c-46e9-94d7-f0cf491ea19a None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:37.242988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:37.243523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:37.243799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:37.244069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:37.244817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] RBAC: Adding query filter params (role.id=96685976a13e4180ba58f74d6dc0cb4f, scope.project.id=536062968e0944c4b04b98b83db365da, user.id=89ea05e1ac114bb3809d7e677df024c6) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:37.245125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:37.246674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da51b03-0cc3-4ad5-9c38-cec71f2225a4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:37.252239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 570/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:37 2022] GET /identity/v3/role_assignments?user.id=89ea05e1ac114bb3809d7e677df024c6&scope.project.id=536062968e0944c4b04b98b83db365da&role.id=96685976a13e4180ba58f74d6dc0cb4f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:39.442237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c493e4-f7b1-4ba3-bc2e-9c2cd7e3034b None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:39.442775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c493e4-f7b1-4ba3-bc2e-9c2cd7e3034b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:39.442775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c493e4-f7b1-4ba3-bc2e-9c2cd7e3034b None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:39.444026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 561/1131] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:39.449537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:39.449937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:39.450092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:39.468385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:39.469564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:39.534581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 571/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:39.812173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c493e4-f7b1-4ba3-bc2e-9c2cd7e3034b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:39.911030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c493e4-f7b1-4ba3-bc2e-9c2cd7e3034b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:39.943428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:39.944039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:39.944341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:39.944612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:39.945712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:39.947133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79905a5-9880-4552-ab6c-c6e014a03ddf None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:39.949284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 562/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:39.955102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:40.035167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:40.051273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea99b0-a892-45af-9839-7e22196704e9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:40.083387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:40.083934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:40.084214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:40.084511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:40.085161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:40.086587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5e15f-8c3b-46df-ade4-058100863aaa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:40.114818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 572/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:17:39 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:42.277850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6654225-7243-4541-9cc5-bb0d5f52c594 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:42.278382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6654225-7243-4541-9cc5-bb0d5f52c594 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:42.278382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6654225-7243-4541-9cc5-bb0d5f52c594 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:42.279709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 563/1135] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:42.284867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:42.285312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:42.285404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:42.303319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:42.304532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:42.371856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 573/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:42.641189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6654225-7243-4541-9cc5-bb0d5f52c594 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:42.690928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:42.742018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6654225-7243-4541-9cc5-bb0d5f52c594 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:42.775161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:42.775775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:42.776051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:42.776346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:42.777561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:42.778969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d72b3a-fb6f-4442-b472-15706372f401 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:42.780526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 564/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:17:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:42.786304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:42.850059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:42.886236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adfef4-2d64-4530-9514-f025838cde36 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:42.919827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:42.920394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:42.920672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:42.920945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:42.921590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:42.923014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97635a-050b-413f-be83-5dc4e1ae8fc1 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:42.943917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 574/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:17:42 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:17:45.108424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48ebb-8666-4771-a074-9c7dfd80361a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:45.109004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48ebb-8666-4771-a074-9c7dfd80361a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:45.109004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48ebb-8666-4771-a074-9c7dfd80361a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:45.110296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 565/1139] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:45.115551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:45.115965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:45.116094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:45.133901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:45.135060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:45.199306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 575/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:45.493534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df48ebb-8666-4771-a074-9c7dfd80361a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:45.606645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df48ebb-8666-4771-a074-9c7dfd80361a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:45.629198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:45.646848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:45.647471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:45.647750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:45.648021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:45.657761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:45.662485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:45.663636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:45.664977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 566/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:45.670209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:45.720081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:45.768104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82901-c45c-402a-a15c-3c1b2842b465 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:45.800919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:45.801513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:45.801802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:45.802056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:45.802701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:45.802998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:45.804384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:45.812835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 576/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:45.818760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:45.922879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353ab-d2d8-4502-a3b1-bddf21f79a07 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:45.957196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:45.957819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:45.958097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:45.958370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] PATH_INFO: `/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:45.971560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccb769adc8b4ae9b5ce1281ff45d9d2)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:45.973010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:45.974579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 567/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:45 2022] GET /identity/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:45.981461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:46.077241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55701bcf-a0b0-46d3-97e5-95735cc8384e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:46.090719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:46.111138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:46.111737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:46.112009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:46.112291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:46.176670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:46.178112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:46.271285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 577/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:46.277679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:46.310764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:46.380827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f912c-b88e-4c09-b39d-19e565e3af77 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:46.413843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:46.414459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:46.414739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:46.415010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] PATH_INFO: `/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:46.416111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] RBAC: Authorizing `identity:get_project(project_id=9d58cd96a2554d0cac41d66aa482a1d4)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:46.417688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974cecf-dcfe-4209-a880-14b409fc27b4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:46.419286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 568/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:46 2022] GET /identity/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:46.425520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:46.440054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:46.523051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b7b60-201d-46f8-9a89-581e7c116f42 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:46.557274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:46.558227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:46.558557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:46.558835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:46.559550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9d58cd96a2554d0cac41d66aa482a1d4, user.id=1ccb769adc8b4ae9b5ce1281ff45d9d2) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:46.559858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:46.561319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bc6-d6dd-405e-8d08-c48df338a33d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:46.566839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 578/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:46 2022] GET /identity/v3/role_assignments?user.id=1ccb769adc8b4ae9b5ce1281ff45d9d2&scope.project.id=9d58cd96a2554d0cac41d66aa482a1d4&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:48.714725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95a4b-8f3b-40ab-87d5-42c6a3a812b3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:48.715305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95a4b-8f3b-40ab-87d5-42c6a3a812b3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:48.715305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95a4b-8f3b-40ab-87d5-42c6a3a812b3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:48.716641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 569/1147] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:48.721764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:48.722141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:48.722319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:48.740780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:48.741953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:48.807071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 579/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:17:49.082020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c95a4b-8f3b-40ab-87d5-42c6a3a812b3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:49.202050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c95a4b-8f3b-40ab-87d5-42c6a3a812b3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:49.204619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:49.245962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:49.246599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:49.246883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:49.247157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:49.263155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:49.269380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:49.270523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:49.271411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 570/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:49.279745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:49.292475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:49.378120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39375b-746d-4498-951d-ef14d74bc87e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:49.410936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:49.411530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:49.411802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:49.412070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:49.412737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:49.413033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:49.414373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:49.422120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 580/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:49.427940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:49.525360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd400b-aaca-4d83-ac7f-cf23d5dc158c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:49.557742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:49.558325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:49.558602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:49.558871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] PATH_INFO: `/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:49.559968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccb769adc8b4ae9b5ce1281ff45d9d2)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:49.561406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:49.562926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 571/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:49 2022] GET /identity/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:49.568558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:49.635920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:49.666480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f0104-5592-4043-ae84-ed20689b6159 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:49.699406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:49.700025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:49.700326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:49.700599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] PATH_INFO: `/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:49.701706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] RBAC: Authorizing `identity:get_project(project_id=9d58cd96a2554d0cac41d66aa482a1d4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:49.703118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7067d-993c-49c0-a0dd-3a0dbd442d4c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:49.704742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 581/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:49 2022] GET /identity/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:49.710752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:49.739977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:49.808211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265dccb-1113-49b8-ab53-6ba921a4c4d4 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:49.840831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:49.841473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:49.841755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:49.842024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] PATH_INFO: `/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4/users/1ccb769adc8b4ae9b5ce1281ff45d9d2/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:49.843881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=9d58cd96a2554d0cac41d66aa482a1d4, user_id=1ccb769adc8b4ae9b5ce1281ff45d9d2, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:49.845327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67720b02-1f4d-42f7-ba55-863249c09dcd None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:49.857740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 572/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:17:49 2022] PUT /identity/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4/users/1ccb769adc8b4ae9b5ce1281ff45d9d2/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:17:51.966704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7327-2a2b-493a-ba73-8c332d8c857c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:51.967266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7327-2a2b-493a-ba73-8c332d8c857c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:51.967266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7327-2a2b-493a-ba73-8c332d8c857c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:51.968607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 582/1154] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:51.973654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14624086-de75-419d-a5aa-08c6c40551db None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:51.974054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14624086-de75-419d-a5aa-08c6c40551db None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:51.974203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14624086-de75-419d-a5aa-08c6c40551db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:51.992543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-14624086-de75-419d-a5aa-08c6c40551db None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:51.993719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-14624086-de75-419d-a5aa-08c6c40551db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:52.058332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 573/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:52.325105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd7327-2a2b-493a-ba73-8c332d8c857c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:52.424238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd7327-2a2b-493a-ba73-8c332d8c857c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:52.457259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:52.457882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:52.458156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:52.458427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:52.465059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:52.469473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:17:52.474392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:52.475334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:17:52.476516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 583/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:52.482467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14624086-de75-419d-a5aa-08c6c40551db None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:52.503665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:52.577794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14624086-de75-419d-a5aa-08c6c40551db None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:52.610404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:52.610993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:52.611269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:52.611534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:52.612183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:52.612511 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:52.613855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:52.622566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 574/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:52.628493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:52.724597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac71d-b2bf-4c8a-bbba-150e0b635366 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:52.759263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:52.759887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:52.760170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:52.761226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] PATH_INFO: `/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:52.762400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] RBAC: Authorizing `identity:get_user(user_id=1ccb769adc8b4ae9b5ce1281ff45d9d2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:52.763836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:52.765444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 584/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:17:52 2022] GET /identity/v3/users/1ccb769adc8b4ae9b5ce1281ff45d9d2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:52.771152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:52.851035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:52.869284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3ea-1df6-416f-85fb-29eccbcafc0b None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:52.902869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:52.903463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:52.903743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:52.904016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:52.968943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:52.970358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:53.046928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:53.065191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 575/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:53.071056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:53.135737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:53.169274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad362e63-8e09-4488-9b9a-180e4627a51c None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:53.201275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:53.201885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:53.202158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:53.202426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] PATH_INFO: `/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:53.203511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] RBAC: Authorizing `identity:get_project(project_id=9d58cd96a2554d0cac41d66aa482a1d4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:53.204954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb2ac0-1d34-4e57-8948-dec16e94c4be None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:53.206542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 585/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:17:53 2022] GET /identity/v3/projects/9d58cd96a2554d0cac41d66aa482a1d4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:53.212680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:53.308804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3117d1-b614-4794-87b9-9d4adfbabfb5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:53.340775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:53.341359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:53.341634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:53.341901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:53.342606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9d58cd96a2554d0cac41d66aa482a1d4, user.id=1ccb769adc8b4ae9b5ce1281ff45d9d2) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:53.342909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:53.344324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43e4f5-ede5-4519-a292-a6a420a9fc19 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:53.349791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 576/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:17:53 2022] GET /identity/v3/role_assignments?user.id=1ccb769adc8b4ae9b5ce1281ff45d9d2&scope.project.id=9d58cd96a2554d0cac41d66aa482a1d4&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:55.540242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2866dd-b569-4fee-a042-7a1ce8354a96 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:55.540796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2866dd-b569-4fee-a042-7a1ce8354a96 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:55.540796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2866dd-b569-4fee-a042-7a1ce8354a96 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:55.542980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 586/1162] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:55.548192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:55.548606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:55.548777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:55.567056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:55.568233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:55.609454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:55.634391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 577/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:55.901886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2866dd-b569-4fee-a042-7a1ce8354a96 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:56.003146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2866dd-b569-4fee-a042-7a1ce8354a96 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:56.035808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:56.036439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:56.036722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:56.036994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:56.039513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:56.104660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:56.106119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:56.200454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 587/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:17:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:17:56.206331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:56.304807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16869cef-592d-411f-bc15-df11b563cc32 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:56.338799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:56.339505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:56.339785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:56.340062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:56.346465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:56.347890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-df3fbb48-c02d-4b98-a4de-6a4dac0c1dc2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:17:56.358975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:17:56.360269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:17:56.361380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 578/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:17:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:17:56.366129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:56.406115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:56.470425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845ffd2-a483-45ee-b2eb-b9f1b9c7edd3 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:56.503102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:56.503698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:56.503973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:56.504241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:56.504892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:17:56.505190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:56.506542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770de60-0b47-4d83-b5a3-0b95453c1e73 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:56.514153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 588/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 05:17:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:17:58.620104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48805-e9ed-4540-9ace-487c82b1ab7b None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:58.620664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48805-e9ed-4540-9ace-487c82b1ab7b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:58.620664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48805-e9ed-4540-9ace-487c82b1ab7b None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:58.621909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 579/1167] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:17:58.627164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d815-15c8-4ebc-a4da-c3444a5af626 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:58.627582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d815-15c8-4ebc-a4da-c3444a5af626 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:58.627688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d815-15c8-4ebc-a4da-c3444a5af626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:58.645988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f121d815-15c8-4ebc-a4da-c3444a5af626 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:17:58.647163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f121d815-15c8-4ebc-a4da-c3444a5af626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:17:58.711343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 589/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:17:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:17:58.987074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48805-e9ed-4540-9ace-487c82b1ab7b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:17:59.024270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:17:59.088665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48805-e9ed-4540-9ace-487c82b1ab7b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:17:59.121122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:17:59.121724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:17:59.122002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:17:59.122273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:17:59.122935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:17:59.124327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095888a-e2a2-4326-9d4b-b61315f367b5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:17:59.140312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 580/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Mon Sep 26 05:17:58 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:18:01.290147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bf3f-41e3-4784-ac65-0be36b49d0a2 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:01.290688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bf3f-41e3-4784-ac65-0be36b49d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:01.290688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185bf3f-41e3-4784-ac65-0be36b49d0a2 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:01.292800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 590/1170] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:18:01.298088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:01.298482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:01.298627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:01.316989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:18:01.318155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:18:01.385896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 581/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:18:01.651696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185bf3f-41e3-4784-ac65-0be36b49d0a2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:01.679457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:01.750534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185bf3f-41e3-4784-ac65-0be36b49d0a2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:01.784018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:01.784661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:01.784936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:01.785203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] PATH_INFO: `/v3/domains/bb520dc95c6f4b7f998735407b53d49c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:01.795896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb520dc95c6f4b7f998735407b53d49c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:01.797425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6716798-8566-428c-8efc-3b556aaaf9f5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:01.798996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 591/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Sep 26 05:18:01 2022] GET /identity/v3/domains/bb520dc95c6f4b7f998735407b53d49c => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:01.804768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:01.830118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:01.901069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35ab9d-7d55-46f9-a318-005d5942b593 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:01.933626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:01.934160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:01.934435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:01.934704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:01.935354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:01.936797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc291f36-7884-41e2-a79d-dd4e67072d8e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:01.962227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 582/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:18:01 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:18:04.111838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daa5f7-279a-41d6-91d8-b5f405be717f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:04.112260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daa5f7-279a-41d6-91d8-b5f405be717f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:04.112411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daa5f7-279a-41d6-91d8-b5f405be717f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:04.113746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 592/1174] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:18:04.119288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:04.119700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:04.119803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:04.137399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:18:04.138558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:18:04.201341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 583/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:18:04.476661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daa5f7-279a-41d6-91d8-b5f405be717f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:04.561393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:04.574826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daa5f7-279a-41d6-91d8-b5f405be717f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:04.606807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:04.607410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:04.607683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:04.607949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] PATH_INFO: `/v3/domains/bb520dc95c6f4b7f998735407b53d49c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:04.609056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb520dc95c6f4b7f998735407b53d49c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:04.610473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ce4bd-0583-4b7e-99ed-4a95ad56f66d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:04.611996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 593/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Sep 26 05:18:04 2022] GET /identity/v3/domains/bb520dc95c6f4b7f998735407b53d49c => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:04.618115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:04.627949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:04.714417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f9394-ab8f-4415-98f9-343c2fce45f9 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:04.747230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:04.747781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:04.748055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:04.748350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:04.749000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:04.750401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46a66c-407c-4eed-b867-ae953579114f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:04.770375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 584/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:18:04 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:18:06.936279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0684-2def-4e76-b19e-06bfb494d0e1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:06.936838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0684-2def-4e76-b19e-06bfb494d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:06.936838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0684-2def-4e76-b19e-06bfb494d0e1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:06.938136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 594/1178] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:18:06.943423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:06.943825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:06.943962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:06.961571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:18:06.963245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:18:07.018972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:07.028509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 585/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:18:07.295347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0684-2def-4e76-b19e-06bfb494d0e1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:07.393587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0684-2def-4e76-b19e-06bfb494d0e1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:07.425645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:07.426248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:07.426520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:07.426784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:07.437062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:18:07.441734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:18:07.442843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:18:07.444062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 595/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:18:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:18:07.449294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:07.545171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04deaf4f-3da3-45b4-a73b-4a6b5e02ad62 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:07.578327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:07.578864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:07.579142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:07.579410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:07.580064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:18:07.580396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:07.581758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:07.590295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 586/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:18:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:07.596002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:07.613905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:07.693390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50099b46-2350-4592-9af5-735d3d706be4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:07.725660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:07.726277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:07.726554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:07.726824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] PATH_INFO: `/v3/users/435aef01abff4e6f89e028e72906652b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:07.739681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] RBAC: Authorizing `identity:get_user(user_id=435aef01abff4e6f89e028e72906652b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:07.741486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:07.743453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 596/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:18:07 2022] GET /identity/v3/users/435aef01abff4e6f89e028e72906652b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:07.750364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:07.804201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:07.846990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4a7c-11a4-4e60-9d17-6119d568a0c0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:07.879438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:07.880032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:07.880333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:07.880604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:07.944576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:07.945994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:07.997754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:08.040689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 587/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:18:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:18:08.046595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:08.122645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:08.147024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef3db-3391-42bd-83c9-58c41aae6600 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:08.179997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:08.180647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:08.180922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:08.181859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] PATH_INFO: `/v3/projects/402452004d4540bc8a2bf476da396075` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:08.182956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] RBAC: Authorizing `identity:get_project(project_id=402452004d4540bc8a2bf476da396075)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:08.184411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53844e1-3d57-40b3-b75c-fdeac99b8b08 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:08.186028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 597/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:18:08 2022] GET /identity/v3/projects/402452004d4540bc8a2bf476da396075 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:08.192169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:08.286858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6067710-993e-4ee2-a027-0d8751801149 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:08.319850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:08.320467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:08.320747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:08.321020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:08.321760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=402452004d4540bc8a2bf476da396075, user.id=435aef01abff4e6f89e028e72906652b) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:18:08.322068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:08.323479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ada1b1-a76e-457b-9a53-8db088397c1f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:08.328643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 588/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:18:08 2022] GET /identity/v3/role_assignments?user.id=435aef01abff4e6f89e028e72906652b&scope.project.id=402452004d4540bc8a2bf476da396075&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:10.423864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784724e-2a98-49da-98c6-a838f0d3a953 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:10.424382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784724e-2a98-49da-98c6-a838f0d3a953 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:10.424382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784724e-2a98-49da-98c6-a838f0d3a953 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:10.425697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 598/1186] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:18:10.430856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:10.431876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:10.432094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:10.449873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:18:10.451042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:18:10.464550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:10.517415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 589/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:18:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:18:10.785673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784724e-2a98-49da-98c6-a838f0d3a953 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:10.883811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784724e-2a98-49da-98c6-a838f0d3a953 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:10.917990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:10.918603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:10.918883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:10.919156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:10.928916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:18:10.933734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:18:10.934918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:18:10.935964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 599/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:18:10.940863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:11.036612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e877d47-8090-46aa-86e6-9bc09c8ae17f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:11.069435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:11.070038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:11.070319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:11.070594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:11.071252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:18:11.072125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:11.073574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:11.081547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 590/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:11.087877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:11.091285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:11.185878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d355f-2f20-49bd-9a6d-5c8bda7740e4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:11.219579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:11.220202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:11.220509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:11.220783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] PATH_INFO: `/v3/users/435aef01abff4e6f89e028e72906652b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:11.223078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] RBAC: Authorizing `identity:get_user(user_id=435aef01abff4e6f89e028e72906652b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:11.224534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:11.226113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 600/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:18:11 2022] GET /identity/v3/users/435aef01abff4e6f89e028e72906652b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:11.231913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:11.247467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:11.329655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5e6d2-6527-4571-b75e-d0566aa383e7 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:11.361833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:11.362453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:11.362733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:11.363006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] PATH_INFO: `/v3/projects/402452004d4540bc8a2bf476da396075` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:11.364111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] RBAC: Authorizing `identity:get_project(project_id=402452004d4540bc8a2bf476da396075)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:11.365574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a522a-eff1-4068-b78b-4bcca9a666c9 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:11.367454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 591/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:18:11 2022] GET /identity/v3/projects/402452004d4540bc8a2bf476da396075 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:11.373506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:11.442026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:11.473599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f68c4-49e5-4356-b353-2c621459ccdc None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:11.508182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:11.508865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:11.509721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:11.509989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] PATH_INFO: `/v3/projects/402452004d4540bc8a2bf476da396075/users/435aef01abff4e6f89e028e72906652b/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:11.511884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] RBAC: Authorizing `identity:create_grant(project_id=402452004d4540bc8a2bf476da396075, user_id=435aef01abff4e6f89e028e72906652b, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:11.513411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e20d59-8928-4d30-bc30-df5ebe60d027 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:11.526661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 601/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:18:11 2022] PUT /identity/v3/projects/402452004d4540bc8a2bf476da396075/users/435aef01abff4e6f89e028e72906652b/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:18:13.647814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d3f9-473b-4be3-b690-e2a809b2fe9f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:13.648362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d3f9-473b-4be3-b690-e2a809b2fe9f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:13.648362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d3f9-473b-4be3-b690-e2a809b2fe9f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:13.649788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 592/1193] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:18:13.655188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:13.655567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:13.655731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:13.673942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:18:13.675098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:18:13.738912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 602/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:18:14.009654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d3f9-473b-4be3-b690-e2a809b2fe9f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.090584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:14.110925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d3f9-473b-4be3-b690-e2a809b2fe9f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:14.143345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:14.143963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:14.144235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:14.144530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:14.155227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:18:14.159909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:18:14.161054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:18:14.161908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 593/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:18:14.167177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.224307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:14.264610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd10a-fa81-428b-abd5-29fadc869468 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:14.297138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:14.297735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:14.298010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:14.298279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:14.298927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:18:14.299225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:14.300611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:14.308568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 603/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:14.314505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.418342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec69-4b96-4f59-8c60-37c25631ca47 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:14.458651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:14.459273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:14.459549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:14.459820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] PATH_INFO: `/v3/users/435aef01abff4e6f89e028e72906652b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:14.460951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] RBAC: Authorizing `identity:get_user(user_id=435aef01abff4e6f89e028e72906652b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:14.462668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:14.464885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 594/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/users/435aef01abff4e6f89e028e72906652b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:14.470581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.573484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04196e7-f989-41e5-8392-e2b943f85247 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:14.594384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:14.605952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:14.606552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:14.606827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:14.607098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:14.672160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:14.673626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:14.769376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 604/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:18:14.775762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.786806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:14.875248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f65db-0d9d-4e59-ba35-8314a141f7fe None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:14.907090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:14.907703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:14.907978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:14.908268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] PATH_INFO: `/v3/projects/402452004d4540bc8a2bf476da396075` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:14.909386 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] RBAC: Authorizing `identity:get_project(project_id=402452004d4540bc8a2bf476da396075)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:14.910799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139c6ad-6cee-4189-bd26-2bda870ea639 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:14.912534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 595/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/projects/402452004d4540bc8a2bf476da396075 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:14.919499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:18:14.942612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:15.015116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c713cd-dc4e-4915-ade9-b51da7398944 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:18:15.047764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:18:15.048376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:18:15.048648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:18:15.048913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:18:15.049615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=402452004d4540bc8a2bf476da396075, user.id=435aef01abff4e6f89e028e72906652b) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:18:15.049921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:18:15.051319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4c49e-e0f9-473c-950f-36d9bf0bb0ac None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:18:15.057452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 605/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:18:14 2022] GET /identity/v3/role_assignments?user.id=435aef01abff4e6f89e028e72906652b&scope.project.id=402452004d4540bc8a2bf476da396075&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:18:24.903084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=62 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:18:25.054323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=57 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:11.102568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c92ff-4eba-4403-8662-1bd37ea5271d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:11.103211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c92ff-4eba-4403-8662-1bd37ea5271d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:11.103211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c92ff-4eba-4403-8662-1bd37ea5271d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:11.107590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 596/1201] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:11.113488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:11.114118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:11.114118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:11.135411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:11.136581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:11.226316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 606/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:19:11.524012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c92ff-4eba-4403-8662-1bd37ea5271d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:11.653014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c92ff-4eba-4403-8662-1bd37ea5271d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:11.694018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:11.695156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:11.695435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:11.695711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:11.710016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:11.711703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:11.730601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:11.731769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:11.732973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:11.732973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:11.732973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:11.732973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 597/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:11.740785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:11.898233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1a2d9-3d61-4f08-bfcb-7618e2c8acc9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:11.938088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:11.948069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:11.949090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:11.949657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:11.950293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:11.951167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:11.951697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:11.953612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0bed3-6f8d-4cac-b52e-9df1c90a872a None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:11.969524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 607/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:11.976181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:12.077148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:12.155587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8987-d35d-417c-aff8-d1aabe302ad8 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:12.199112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:12.199112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:12.199112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:12.199112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:12.199561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:12.200326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c15ff-692a-4c23-9bc8-02879632df2b None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:12.230261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 598/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 05:19:11 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:19:14.794203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec5079-1de7-41c6-b60f-cd25625a2f41 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:14.794960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec5079-1de7-41c6-b60f-cd25625a2f41 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:14.798733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec5079-1de7-41c6-b60f-cd25625a2f41 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:14.800154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 608/1206] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:14.808569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:14.809147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:14.809147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:14.831527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:14.832743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:14.914489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 599/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:15.209156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec5079-1de7-41c6-b60f-cd25625a2f41 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:15.330276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec5079-1de7-41c6-b60f-cd25625a2f41 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:15.366900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:15.367401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:15.367719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:15.367931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:15.380186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:15.386328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:15.387480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:15.388404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:15.392443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 609/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:19:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:15.394001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:15.445522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:15.522115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42883-929a-4134-a603-2cfa46bb9230 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:15.557704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:15.558315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:15.558642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:15.558887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:15.559522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:15.559824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:15.561230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:15.569491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 600/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:19:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:15.575974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:15.639690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:15.699592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65099-6eb4-4f36-b78f-b84bf8876d96 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:15.739932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:15.740618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:15.741131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:15.741401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:15.752933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:15.754493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:15.770431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:15.773063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:15.774325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:15.774325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:15.774325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:15.774325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 610/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:15.780129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:15.891459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:15.894020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66370-2cf1-4288-ae78-d93d271a826c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:15.938510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:15.939149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:15.939430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:15.939703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:15.940448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:15.940754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:15.942183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:15.960109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 601/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:15.961463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:16.078649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c4d2d-71e5-49ee-9de8-48a454cae0f8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:16.080806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:16.123089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:16.124023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:16.124550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:16.125033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:16.133548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:16.135411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:16.148424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:16.149501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:16.150584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:16.153240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 611/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:16.158967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:16.272681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedbcd2-c505-4653-a179-9208cf8a5b54 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:16.312644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:16.313709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:16.314261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:16.314747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:16.315687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:16.316201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:16.317972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:16.340137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 602/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:16.346014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:16.460917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee56bbd-fa68-4d9c-af6a-e51b24686075 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:16.500486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:16.502150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:16.502150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:16.502150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:16.532736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:16.578749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:16.580196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:16.691483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 612/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:19:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:19:16.697260 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:16.717865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:16.826273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b35fbb-6f4b-4622-b27e-4fc2c2990d93 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:16.865473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:16.866178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:16.866594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:16.866859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:16.877996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:16.879562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:16.896845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:16.898442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:16.899549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:16.899549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:16.899549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:16.899549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 603/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:16.904749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:16.970696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:17.017161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54f1e-03a6-41be-b669-fa89dae1f88b None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:17.054465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:17.055079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:17.055354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:17.055622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:17.056276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:17.056601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:17.058085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:17.071914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 613/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:17.078379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:17.158418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:17.194980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac62f22-daa2-4571-b572-bbe43c619065 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:17.233238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:17.233793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:17.234071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:17.234336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:17.242612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:17.244544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:17.257677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:17.258699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:17.259627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 604/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:17.264635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:17.381124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31449391-4fd4-419d-bf61-90f34742f981 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:17.407997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:17.422140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:17.422767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:17.423040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:17.423443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:17.424258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:17.424762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:17.426412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f62c33-ab52-4caa-b733-3206237a0828 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:17.440949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 614/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:17.447003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:17.564058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9336590-932d-4f85-b978-ec343d04d583 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:17.593793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:17.602191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:17.602823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:17.603104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:17.603377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:17.604108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=35bda10601d04f22a509cfb45e19c6ae) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:17.604448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:17.605955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17379f5-c4f3-4462-9191-e485d121b725 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:17.613323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 605/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:19:17 2022] GET /identity/v3/role_assignments?user.id=35bda10601d04f22a509cfb45e19c6ae&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:19.947326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e42-80d1-455d-8d9d-103eb49d1cc6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:19.947880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e42-80d1-455d-8d9d-103eb49d1cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:19.947880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e42-80d1-455d-8d9d-103eb49d1cc6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:19.949603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 615/1220] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:19.955594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:19.956049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:19.956190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:19.974605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:19.975815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:20.053165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 606/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:19:20.374933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76e42-80d1-455d-8d9d-103eb49d1cc6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:20.487778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76e42-80d1-455d-8d9d-103eb49d1cc6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:20.527053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:20.527682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:20.527954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:20.528226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:20.539634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:20.544679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:20.545864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:20.546873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 616/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:20.552290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:20.659548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff525c67-d086-46f0-bf63-31a33033301d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:20.684217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:20.695989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:20.696622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:20.696901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:20.697208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:20.697868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] RBAC: Adding query filter params (name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:20.698164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:20.699728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:20.707863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 607/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:19:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:20.714173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:20.763921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:20.826681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b2cf-aabb-4a09-8380-88e11d72bc64 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:20.869389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:20.870011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:20.870288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:20.870556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:20.881982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:20.883608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:20.903848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:20.905121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:20.906268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:20.906268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:20.906268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:20.906268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 617/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:20.911798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.032610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e8876-6db7-4ca2-85d2-b728da56a069 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.069592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.070212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.070488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.070760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.071399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:21.071745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.073232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.087838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 608/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:21.094403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.197682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:21.206045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85d0fc-5207-44af-ac46-0a922fb675e9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.242822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.243582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.243924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.244242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.253322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.255232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:21.268340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:21.269451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:21.270567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 618/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:21.276241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.326349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:21.385204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3f49-1d1a-4dd9-826c-77f3f7dab07c None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.419738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.420366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.420647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.420917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.421642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:21.421945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.423419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.443896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 609/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:21.450935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.559247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddb5e2-b339-448a-aa14-46db1e42f238 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.597333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.598276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.598732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.599282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.611441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.613168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:21.631529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:21.632725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:21.633906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:21.633906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:21.633906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:21.633906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 619/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:21.638809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.754606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff6906-32d4-4c03-b13d-db0bf7c14ba2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.758279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:21.800567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.801203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.801504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.801747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.803209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:21.803496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.804888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.816339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 610/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:21.822539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:21.831814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:21.927910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26fbe-6443-4b05-8073-8717b82e54ad None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:21.965064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:21.965849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:21.966157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:21.966546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:21.973149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:21.975251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:21.988153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:21.989572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:21.990610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 620/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:21.995880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:22.120940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041ca59-deda-4652-83ef-5d4b20db2d1a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:22.159211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:22.159873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:22.160124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:22.160462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:22.161221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:22.162138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:22.163933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de256-2bb4-454c-aec4-2e505f2277b8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:22.178623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 611/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:22.185694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:22.263495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:22.301668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913834-2e03-4500-9e4f-bd21eca19220 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:22.338531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:22.339226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:22.339596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:22.339873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/35bda10601d04f22a509cfb45e19c6ae/roles/c7e31a27698f4b16bb1a6a8847881b1c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:22.356669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=35bda10601d04f22a509cfb45e19c6ae, role_id=c7e31a27698f4b16bb1a6a8847881b1c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:22.358313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564ce66-67bb-4984-bbef-03a93fadf004 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:22.376305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 621/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:19:22 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/35bda10601d04f22a509cfb45e19c6ae/roles/c7e31a27698f4b16bb1a6a8847881b1c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:19:24.571201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bce61-b64a-43fe-937f-dace0788efbc None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:24.571729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bce61-b64a-43fe-937f-dace0788efbc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:24.571822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bce61-b64a-43fe-937f-dace0788efbc None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:24.573268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 612/1233] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:24.578610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:24.579086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:24.579179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:24.600035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:24.601294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:24.671789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 622/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:24.957445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bce61-b64a-43fe-937f-dace0788efbc None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:24.986460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:25.074730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bce61-b64a-43fe-937f-dace0788efbc None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:25.120789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:25.121486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:25.121796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:25.122197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:25.134721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:25.139574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:25.141032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:25.141922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 613/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 05:19:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:25.148231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:25.200581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:25.261119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8fc4-7813-4a30-87e8-365c4e4a7f38 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:25.301094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:25.301728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:25.302060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:25.302332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:25.303038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:25.303347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:25.305207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:25.314175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 623/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 05:19:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:25.321782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:25.417061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:25.433330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b6649-c330-49a2-94d6-5b6166616e50 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:25.468621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:25.469238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:25.469510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:25.469776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:25.479686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:25.481148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:25.498720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:25.505644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:25.507068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:25.507068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:25.507068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:25.507068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 614/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:25.507369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:25.623967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f712-9ae5-4f50-af78-0a951a359e06 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:25.634981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:25.664688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:25.665301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:25.665585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:25.665849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:25.666479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:25.666822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:25.668195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:25.681471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 624/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:25.687606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:25.807739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e9fd4-a87e-438b-9f32-8b68f984f1ee None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:25.844526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:25.845251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:25.845555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:25.845864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:25.849138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:25.854677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:25.856369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:25.870117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:25.871159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:25.872145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 615/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:25.877752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:25.988311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055cf43-59bb-47ac-a594-38ffc0ca59f5 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:26.030551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:26.031286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:26.031599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:26.031906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:26.032760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:26.033135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:26.034760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:26.055401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 625/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:26.061486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:26.169761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb119cfe-092b-4267-9cfb-8ef244810e6e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:26.207690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:26.208342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:26.208621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:26.208886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:26.273646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:26.280633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:26.282125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:26.402767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 616/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:19:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:19:26.404043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:26.433104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:26.514884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2eacbf-c2b5-4b32-ba0e-2b83c199f0b4 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:26.553886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:26.554524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:26.554800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:26.555076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:26.566683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:26.568631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:26.585994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:26.587111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:26.588059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:26.588059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:26.588059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:26.588059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 626/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:26.593460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:26.701362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:26.708709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68972a7f-88c0-4c71-a3d6-8f968431c55a None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:26.747068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:26.747682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:26.747959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:26.748327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:26.748981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:26.749281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:26.750798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:26.763100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 617/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:26.768969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:26.862772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:26.888880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30dac-fe10-4257-ae63-6cfd6427fd2d None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:26.927939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:26.928691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:26.929111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:26.929462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:26.936642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:26.938233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:26.951088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:26.952292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:26.958887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 627/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:26.959506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:27.068844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdbf8-a9dd-4d11-8962-30eee519fd05 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:27.113122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:27.114038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:27.114475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:27.114790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:27.115638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:27.116067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:27.117520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9551-2ec6-416a-a982-d931811cb18e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:27.134809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 618/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:27.141514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:27.254468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0919b-77a3-4a6b-ae25-dbd7edb91687 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:27.290828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:27.291431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:27.291702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:27.291966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:27.292717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] RBAC: Adding query filter params (role.id=c7e31a27698f4b16bb1a6a8847881b1c, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=35bda10601d04f22a509cfb45e19c6ae) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:27.293022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:27.294482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05e6b6-7572-4f48-8a09-783a1d1e8180 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:27.300618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 628/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:19:27 2022] GET /identity/v3/role_assignments?user.id=35bda10601d04f22a509cfb45e19c6ae&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=c7e31a27698f4b16bb1a6a8847881b1c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:29.656589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86c4d-8f28-4c2b-86ff-20a92e50d39d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:29.657169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86c4d-8f28-4c2b-86ff-20a92e50d39d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:29.657169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86c4d-8f28-4c2b-86ff-20a92e50d39d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:29.658453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 619/1247] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:29.663939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:29.664419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:29.664530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:29.670844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:29.685573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:29.686764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:29.756191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 629/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:30.029042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac86c4d-8f28-4c2b-86ff-20a92e50d39d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:30.049155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:30.144729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac86c4d-8f28-4c2b-86ff-20a92e50d39d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:30.180950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:30.181817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:30.182091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:30.182485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:30.194818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:30.200791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:30.204458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:30.205532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 620/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:30.208509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:30.322583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bb728-d3c6-42be-b225-f8711660637c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:30.353841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:30.359076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:30.359679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:30.359954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:30.360221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:30.360898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:30.361197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:30.362549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:30.371173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 630/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:30.377337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:30.466188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:30.493621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823224-e0a2-40aa-b1d1-461e38d1a0e0 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:30.529637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:30.530369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:30.530708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:30.530982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:30.542862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:30.544519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:30.562391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:30.563562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:30.564725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:30.564725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:30.564725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:30.564725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 621/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:30.571175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:30.689723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb776695-6ec1-420c-a58e-366ab9598082 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:30.733484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:30.734121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:30.734398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:30.734666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:30.735311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:30.735613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:30.737037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:30.751504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 631/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:30.758568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:30.885167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb3955-846e-43ea-ad7e-b28d938ab5c2 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:30.917853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:30.927123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:30.927802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:30.928079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:30.928385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:30.936880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:30.938336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:30.950509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:30.951718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:30.952940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 622/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:30.960435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:31.018173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:31.067288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d193e2c-ed47-4319-9955-c80a2b54d656 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:31.106281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:31.106908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:31.107186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:31.107460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:31.108199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:31.108537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:31.109990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:31.135042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 632/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:19:31.136960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:31.242409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44909f00-8096-42b8-8fb5-6edbc3c0ecf5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:31.275127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:31.275735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:31.276015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:31.276310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:31.336126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:31.353622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:31.355731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:31.463085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 623/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:19:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:19:31.468975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:31.564841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3b176-f2b0-41b2-b22b-e516adf6f60e None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:31.577376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:31.597975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:31.599104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:31.599380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:31.599648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:31.609224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:31.610649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:31.625438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:31.626559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:31.627709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:31.627709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:31.627709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:31.627709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 633/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:31.632188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:31.730633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:31.747709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee81a65-2133-46ff-98ce-6a054a57dd01 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:31.794166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:31.794892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:31.795285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:31.795606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:31.796322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:31.796631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:31.798209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:31.810003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 624/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:31.815688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:31.912329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae21c9-93e7-4d8e-bb0f-a11bcf19eab6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:31.943808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:31.944456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:31.944718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:31.944984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:31.951343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:31.952806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:31.964155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:31.965234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:31.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 634/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:31.971125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:32.071348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf716-d024-4bde-afb2-7bed8a276768 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:32.104153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:32.105094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:32.106668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:32.106668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:32.106668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:32.106934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:32.108272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d322a-c5c5-43cc-ab11-b1d1f4b91f9a None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:32.120442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 625/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:32.126372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:32.152862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:32.224482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead21f9-408f-4493-95f0-292795a14da6 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:32.256735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:32.257322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:32.257599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:32.257868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:32.258574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=35bda10601d04f22a509cfb45e19c6ae) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:32.258876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:32.260313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d7d8-e70e-459e-80e6-0fd635e8edef None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:32.265439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 635/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:19:32 2022] GET /identity/v3/role_assignments?user.id=35bda10601d04f22a509cfb45e19c6ae&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:34.367278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acb15c-9729-4c3f-8b57-d012cf12d2d3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:34.367795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acb15c-9729-4c3f-8b57-d012cf12d2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:34.367795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acb15c-9729-4c3f-8b57-d012cf12d2d3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:34.369075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 626/1261] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:34.374060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:34.374490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:34.374605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:34.393179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:34.394346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:34.458354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 636/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:34.724638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acb15c-9729-4c3f-8b57-d012cf12d2d3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:34.735286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:34.838632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acb15c-9729-4c3f-8b57-d012cf12d2d3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:34.873121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:34.873737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:34.874012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:34.874279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:34.884176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:34.889159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:34.890362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:34.891418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 627/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:19:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:19:34.897250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:34.915498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:34.993695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8f0e-f7b8-4379-bbb2-0b144baba382 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.025636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.026220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.026491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.026760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.027396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:35.027689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.029052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.037296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 637/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:19:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:35.043168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.107262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:35.141563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b32d-f838-4661-8728-4c6f99b13e88 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.173650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.174261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.174538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.174808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.185072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.186500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:35.200699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:35.201840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:35.202930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:35.202930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:35.202930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:35.202930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 628/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:35.208231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.287283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:35.306384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa84ec1-7dde-47a4-80a9-bb2ab822d4cf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.339232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.339821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.340091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.340389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.341016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:35.341306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.342651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.354136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 638/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:35.359743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.457759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896be5e-5967-44b1-b30d-c41e0cfb6f4f None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.476472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:35.490731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.491345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.491619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.491887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.499044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.500499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:35.511869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:35.513064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:35.514033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 629/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:35.518805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.614404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6627d-dd8d-4a52-822f-69dc8cda8ac8 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.647271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.647858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.648131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.648431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.649142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:35.649442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.650850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.659096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:35.668241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 639/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:35.673967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.779088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012265-56b6-42e3-afe9-4183c5a70d9e None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.812802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.813422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.813700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.813969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.824208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.825664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:35.839295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:35.840476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:35.841582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:35.841582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:35.841582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:35.841582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 630/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:35.846117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:35.942108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e75ee3-2fba-494a-a58c-d5e24e7e7ffe None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:35.974761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:35.975353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:35.975626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:35.975895 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:35.976546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:35.976844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:35.978190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:35.989732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 640/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:35.995239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:36.025396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:36.091939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c0a6-68ba-4f87-b22d-149aa22bc014 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:36.125164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:36.125783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:36.126060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:36.126328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:36.132863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:36.134295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:36.146820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:36.147884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:36.148855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 631/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:36.153706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:36.212394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:36.254490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4488a-6fa1-481f-81fe-8504fb084bfd None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:36.290083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:36.290685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:36.290960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:36.291231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:36.291911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:36.292213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:36.293745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed089b-8aa4-4fd8-90bd-2bf6419ab29f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:36.306169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 641/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:36.312303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:36.408770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:36.423302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbffd4-758d-4535-b2b2-defda26f2162 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:36.463978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:36.464674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:36.464842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:36.465110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] PATH_INFO: `/v3/projects/9f32b9319d9146eb847509775ea6af53/users/35bda10601d04f22a509cfb45e19c6ae/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:36.467000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f32b9319d9146eb847509775ea6af53, user_id=35bda10601d04f22a509cfb45e19c6ae, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:36.468508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3382b-f8dc-4a0e-9014-bd046d9569d8 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:36.482487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 632/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 05:19:36 2022] PUT /identity/v3/projects/9f32b9319d9146eb847509775ea6af53/users/35bda10601d04f22a509cfb45e19c6ae/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:19:38.689871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373ac-f818-4e08-94c7-a79f5e14d0d1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:38.690404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373ac-f818-4e08-94c7-a79f5e14d0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:38.690404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373ac-f818-4e08-94c7-a79f5e14d0d1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:38.692701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 642/1274] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:38.697926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:38.698307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:38.698478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:38.716238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:38.717450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:38.786992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 633/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:39.066693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a373ac-f818-4e08-94c7-a79f5e14d0d1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.167352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a373ac-f818-4e08-94c7-a79f5e14d0d1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.185057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:39.202543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.203163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.203441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.203710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.214689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:19:39.219473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:39.220604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:39.221497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 643/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:39.227282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.304960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:39.322694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd98fa7-36be-4d54-8f92-1b10cda562b4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.355645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.356230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.356542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.356811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.357459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:39.357753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:39.359103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:39.367173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 634/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:39.372886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.468212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263736fe-21e6-4ff4-a51f-5b0f8d18b3c1 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.500945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.501549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.501820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.502083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.511610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:39.513062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:39.527535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:39.528758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:39.529900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:39.529900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:39.529900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:39.529900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 644/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:39.536614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.631757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe00216-3d7b-43db-94cd-7fa0741e77f5 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.663362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.663946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.664220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.664518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.665134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:39.665427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:39.666766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:39.678150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 635/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:39.683591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.714099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:39.782995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b49f-8c9e-4ad2-8d13-1c4a1e3df965 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.814666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.815276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.815549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.815815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.823371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:39.824821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:19:39.836271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:39.837390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:39.838602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 645/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:39.843164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:39.846948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:39.940059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e499-53ea-4cd4-bb97-9bd0ea2d8453 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:39.972599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:39.973201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:39.973480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:39.973749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:39.974462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:39.974762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:39.976163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:39.992854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 636/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:39.998684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:40.086721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:40.096829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb276d8e-cc72-4145-bc57-1f7f8f973dba None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:40.130282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:40.130876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:40.131156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:40.131424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:40.196236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:40.197691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:40.282499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:40.293439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 646/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 05:19:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:19:40.299176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:40.358790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:40.395954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32235-8c33-4764-8aee-565b33fc0516 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:40.428762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:40.429366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:40.429637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:40.429900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:40.439373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:40.440817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:19:40.454621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:40.455576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:19:40.456602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:19:40.456602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:19:40.456602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:40.456602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 637/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:40.461470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:40.559852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7bb3-1ed4-4e86-88d3-fc420d3cadaa None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:40.592278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:40.592871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:40.593144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:40.593412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:40.594038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:40.594336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:40.595691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:40.607091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 647/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 05:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:40.612623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:40.708495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311bd32-0fb9-4b0c-ab81-a78aa92817e1 None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:40.721175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:40.741175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:40.741786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:40.742062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:40.742329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:40.748764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:40.750269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:19:40.761425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:40.762514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: ERROR keystone.server.flask.application Sep 26 05:19:40.763950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 638/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 05:19:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:40.768283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:40.819244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:40.869295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4e3e-418b-4d40-a3f1-941f28e041cf None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:40.901688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:40.902278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:40.902551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:40.902817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:40.903490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:40.903789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:40.905229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2e07d-20d8-4cec-9bac-c2e0bc98ac04 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:40.917262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 648/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 05:19:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:40.923469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:41.021670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841b24-b351-44ea-a39d-d5c413878ece None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:41.054021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:41.054612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:41.054893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:41.055165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:41.055880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] RBAC: Adding query filter params (role.id=e0af1611fa564e7d8d26e3e289f6c5c5, scope.project.id=9f32b9319d9146eb847509775ea6af53, user.id=35bda10601d04f22a509cfb45e19c6ae) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:41.056188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:41.057621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fedf3-5d49-4fad-aa8a-d9e36cdf47ad None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:41.063180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 639/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 05:19:40 2022] GET /identity/v3/role_assignments?user.id=35bda10601d04f22a509cfb45e19c6ae&scope.project.id=9f32b9319d9146eb847509775ea6af53&role.id=e0af1611fa564e7d8d26e3e289f6c5c5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:43.197602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36debe50-d889-402b-b56c-1b9d2b178878 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:43.198114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36debe50-d889-402b-b56c-1b9d2b178878 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:43.198209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36debe50-d889-402b-b56c-1b9d2b178878 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:43.199462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 649/1288] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:43.204601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:43.205040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:43.205140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:43.224155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:43.225365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:43.253651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:43.290450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 640/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:43.560810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36debe50-d889-402b-b56c-1b9d2b178878 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:43.658907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36debe50-d889-402b-b56c-1b9d2b178878 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:43.669231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:43.693294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:43.693908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:43.694183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:43.694450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:43.695018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:43.696421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:43.705245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:43.706462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:43.707391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 650/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:19:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:43.712341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:43.815702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a568-2674-415f-ad6f-9d98f2690972 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:43.848973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:43.849561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:43.849835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:43.850102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:43.850719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:43.851014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:43.852385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a07fc-4632-4517-af46-528595e835b2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:43.856963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 641/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 05:19:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:43.862775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:43.960912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95fa08-3fff-486a-9a8a-c1ef12929f85 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:43.993713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:43.994305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:43.994577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:43.994844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] PATH_INFO: `/v3/services` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:43.995455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:43.995775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:43.997155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f4420-6f4a-43da-81cd-182b9cecddf9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:44.001393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 651/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 05:19:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:46.150078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf4ab-9ff1-420c-80f5-ddc23c827c4f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:46.150525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf4ab-9ff1-420c-80f5-ddc23c827c4f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:46.150630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf4ab-9ff1-420c-80f5-ddc23c827c4f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:46.151933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 642/1293] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:46.157000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e6b5-d8f7-436b-a9b3-22443e54862f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:46.157409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e6b5-d8f7-436b-a9b3-22443e54862f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:46.157544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e6b5-d8f7-436b-a9b3-22443e54862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:46.176466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dc92e6b5-d8f7-436b-a9b3-22443e54862f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:46.177633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc92e6b5-d8f7-436b-a9b3-22443e54862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:46.206383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:46.244098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 652/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:46.507412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf4ab-9ff1-420c-80f5-ddc23c827c4f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:46.539139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:46.607807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf4ab-9ff1-420c-80f5-ddc23c827c4f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:46.639725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:46.640451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:46.640731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:46.640999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:46.641636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:46.642971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9826b-ab77-416b-b2cd-87403e154c05 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:46.656921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 643/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 05:19:46 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:19:48.814722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaedcf1-def3-461d-80e0-b8ecc8e381dd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:48.815299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaedcf1-def3-461d-80e0-b8ecc8e381dd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:48.815299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaedcf1-def3-461d-80e0-b8ecc8e381dd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:48.817141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 653/1296] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:48.822317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:48.822731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:48.822865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:48.843963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:48.845185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:48.915782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 644/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:49.198530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaedcf1-def3-461d-80e0-b8ecc8e381dd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:49.295737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaedcf1-def3-461d-80e0-b8ecc8e381dd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:49.327880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:49.328516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:49.328788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:49.329054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:49.329618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:49.330963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:49.339431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:49.340754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:49.341965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 654/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:19:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:49.346466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:49.366319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:49.446469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b12cdc-f901-48c3-a622-08b9500631be None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:49.480705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:49.481304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:49.481585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:49.481858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:49.482480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] RBAC: Adding query filter params (name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:49.482780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:49.484137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:49.490588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 645/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 05:19:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:49.496600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:49.526955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:49.594124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb6d84-f987-4676-87e2-d15281afe7b9 None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:49.626399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:49.626935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:49.627210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:49.627478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:49.628689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] RBAC: Adding query filter params (interface=public, service_id=6f62b811776d47fe963d362228ba1684, region_id=RegionOne) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:49.628983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:49.630456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad8b40-f513-4438-bbe9-9ccc85d02b98 None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:49.635249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 655/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 05:19:49 2022] GET /identity/v3/endpoints?service_id=6f62b811776d47fe963d362228ba1684&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:49.640799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:49.719273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:49.736725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a88b7-b654-409a-86e2-a6ab613165da None admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:49.770429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:49.771022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:49.771302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:49.771575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:49.772171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:49.773553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaf266-ca88-42d5-b479-a86524e3b277 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:49.780314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 646/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 05:19:49 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:19:51.925860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027149a2-e171-47ed-ba92-a86095a921fd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:51.926390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027149a2-e171-47ed-ba92-a86095a921fd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:51.926390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027149a2-e171-47ed-ba92-a86095a921fd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:51.927709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 656/1302] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:19:51.932858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:51.933272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:51.933415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:51.951071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:19:51.952273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:19:52.016691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 647/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:19:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:19:52.288516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027149a2-e171-47ed-ba92-a86095a921fd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:52.380785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:52.387554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027149a2-e171-47ed-ba92-a86095a921fd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:52.419908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:52.420552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:52.420829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:52.421098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:52.421666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:52.423022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:19:52.431500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:19:52.432676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application with Lock( Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: ERROR keystone.server.flask.application Sep 26 05:19:52.433535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 657/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 05:19:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:19:52.438567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:52.513204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:19:52.536220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02115af-05b7-42fb-84a9-f652d5e74a12 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:52.568828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:52.569376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:52.569659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:52.569935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] PATH_INFO: `/v3/services` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:52.570560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:19:52.570863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:52.572223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19053439-8d3f-492a-83b3-828d58c42bb2 None admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:52.577165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 648/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 05:19:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:19:52.583473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:19:52.679931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a85f8-4c25-4688-bab1-62a67fc56d7f None admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:19:52.712403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:19:52.712987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:19:52.713263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:19:52.713531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:19:52.714177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:19:52.715525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48f337-5af0-432f-bc9c-d8b63dfcc68d None admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:19:52.734829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 658/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:19:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:20:02.574304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=30 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:20:02.730607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:20:56.695704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563db1f-18d3-4595-834a-e1d36b6a72be None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:20:56.696204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563db1f-18d3-4595-834a-e1d36b6a72be None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:20:56.696353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563db1f-18d3-4595-834a-e1d36b6a72be None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:20:56.698438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 649/1307] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:20:56.704640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1403a-b90f-4e0b-a61f-54aa34632824 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:20:56.705127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1403a-b90f-4e0b-a61f-54aa34632824 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:20:56.705247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1403a-b90f-4e0b-a61f-54aa34632824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:20:56.787339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7ce1403a-b90f-4e0b-a61f-54aa34632824 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:20:56.788599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ce1403a-b90f-4e0b-a61f-54aa34632824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:20:56.904291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 659/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:20:57.196825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:20:57.197104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:20:57.197553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:20:57.198935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 650/1309] 10.222.0.49 () {54 vars in 964 bytes} [Mon Sep 26 05:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:20:57.206254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6e3ef-1f7c-42ca-aaff-f650075e4bde None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:20:57.206585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6e3ef-1f7c-42ca-aaff-f650075e4bde None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:20:57.206829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6e3ef-1f7c-42ca-aaff-f650075e4bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:20:57.297076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1fe6e3ef-1f7c-42ca-aaff-f650075e4bde None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:20:57.298269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1fe6e3ef-1f7c-42ca-aaff-f650075e4bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:20:57.320923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:20:57.381739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 660/1310] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Sep 26 05:20:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:20:57.390913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:20:57.392731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:20:57.511609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5ff01-f1a4-4356-9a0c-b7539bb35d8f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:20:57.597719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:20:57.598326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:20:57.598598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:20:57.598862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:20:57.600466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:20:57.659303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:20:57.716647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:20:57.718739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ddc3-3a50-443b-801d-3afaa0e21edd service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:20:57.739016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 651/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:00.204552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:00.205081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:00.205081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:00.206614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 661/1312] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:00.211607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cc8a8-c8d8-4e13-bcc8-4c0bbfb3f33a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:00.211993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cc8a8-c8d8-4e13-bcc8-4c0bbfb3f33a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:00.212157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cc8a8-c8d8-4e13-bcc8-4c0bbfb3f33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:00.298902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-da6cc8a8-c8d8-4e13-bcc8-4c0bbfb3f33a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:00.300103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-da6cc8a8-c8d8-4e13-bcc8-4c0bbfb3f33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:00.398635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 652/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:00.684713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:00.687133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:00.805028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd657b-7e28-4881-902f-960acee820e5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:00.880629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:00.882980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:00.883770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:00.884208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:00.884765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:00.886391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:00.996120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:00.997584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13fe01-e304-4882-92ab-238540dfc3c9 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:01.016648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 662/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:03.493197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:03.493732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:03.493732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:03.495065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 653/1315] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:03.501048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779460-9416-47fa-8742-1fbadd5e99e5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:03.501446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779460-9416-47fa-8742-1fbadd5e99e5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:03.501593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779460-9416-47fa-8742-1fbadd5e99e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:03.565865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bc779460-9416-47fa-8742-1fbadd5e99e5 None None] MFA Rules not processed for user `7d17d179d17e40648c50edcb7eb2560d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:03.567043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bc779460-9416-47fa-8742-1fbadd5e99e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:03.617455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:03.661082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 663/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:03.942234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:03.943765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:03.954077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:04.046532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7b1f-d10d-4873-a9a6-1fef64b3bf60 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:04.122457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:04.123150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:04.123561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:04.123840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:04.125508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:04.187236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:04.234689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:04.236401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac7a67-4428-436e-a1d5-69b172879bcf service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:04.254198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 654/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:06.934346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:06.934891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:06.934891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:06.936352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 664/1318] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:06.941456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231747b-a395-414e-9d5f-6f407988d60b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:06.941882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231747b-a395-414e-9d5f-6f407988d60b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:06.942002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231747b-a395-414e-9d5f-6f407988d60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:07.006436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e231747b-a395-414e-9d5f-6f407988d60b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:07.007613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e231747b-a395-414e-9d5f-6f407988d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:07.102662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 655/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:07.390185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:07.391725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:07.492988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687760d8-6cbb-4cda-aa86-f58ab7c290e1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:07.565304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:07.565906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:07.566181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:07.566449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:07.568024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:07.574770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:07.677522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:07.678878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197c108-e0b1-4ff0-b586-cdfb63d76da0 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:07.697032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 665/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:10.494426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:10.494961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:10.494961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:10.496305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 656/1321] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:10.501523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67e45-5033-4fe3-91ae-1fd886df931a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:10.501958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67e45-5033-4fe3-91ae-1fd886df931a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:10.502180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67e45-5033-4fe3-91ae-1fd886df931a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:10.572287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-94e67e45-5033-4fe3-91ae-1fd886df931a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:10.573559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-94e67e45-5033-4fe3-91ae-1fd886df931a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:10.636152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:10.679203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 666/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:10.964842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:10.967095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:10.987422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:11.074411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59508cf-83ad-491e-89f9-9395f56a5d09 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:11.150000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:11.150608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:11.150882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:11.151153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:11.152760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:11.223418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:11.261935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:11.264166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dab10a-a514-47cd-ad97-b07dc8290d4b service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:11.282336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 657/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:13.506537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:13.507073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:13.507073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:13.508450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 667/1324] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:13.514277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106100-0d8b-4a8c-b7d1-ea7f132ebcf1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:13.514719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106100-0d8b-4a8c-b7d1-ea7f132ebcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:13.514899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106100-0d8b-4a8c-b7d1-ea7f132ebcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:13.581414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ec106100-0d8b-4a8c-b7d1-ea7f132ebcf1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:13.582612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ec106100-0d8b-4a8c-b7d1-ea7f132ebcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:13.690636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 658/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:13.975844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:13.977487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:14.085111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3b80-f214-4947-9403-8bc7724c84cd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:14.163572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:14.164202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:14.164505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:14.164773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:14.166355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:14.183831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:14.276329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:14.277685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448a01-dd3d-4c3c-b358-e60b1e9ae2ee service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:14.296749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 668/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:16.521402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:16.521913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:16.521913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:16.523211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 659/1327] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:16.528451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2559c7-c9e3-4fb9-a596-9f771b4966dd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:16.528854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2559c7-c9e3-4fb9-a596-9f771b4966dd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:16.529002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2559c7-c9e3-4fb9-a596-9f771b4966dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:16.594442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bc2559c7-c9e3-4fb9-a596-9f771b4966dd None None] MFA Rules not processed for user `7d17d179d17e40648c50edcb7eb2560d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:16.595565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bc2559c7-c9e3-4fb9-a596-9f771b4966dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:16.663261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:16.695878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 669/1328] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:21:16.982211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:16.983747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:17.017057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:17.095316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92349410-ce20-4c2b-9482-265b2c82ef07 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:17.168754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:17.169270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:17.169542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:17.169809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:17.171389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:17.248639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:17.281174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:17.282627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e2bd9-2208-4c47-87dd-9f739fb66cf2 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:17.300641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 660/1329] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:21:19.691418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d765f6-ea4c-4f4b-87a8-f4a0d9d00381 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:19.692049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d765f6-ea4c-4f4b-87a8-f4a0d9d00381 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:19.692049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d765f6-ea4c-4f4b-87a8-f4a0d9d00381 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:19.693314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 670/1330] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:19.698514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f4d6-eb7b-48c6-bae8-22d2cb5b4c3c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:19.698917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f4d6-eb7b-48c6-bae8-22d2cb5b4c3c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:19.699071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f4d6-eb7b-48c6-bae8-22d2cb5b4c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:19.765134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5625f4d6-eb7b-48c6-bae8-22d2cb5b4c3c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:19.766335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5625f4d6-eb7b-48c6-bae8-22d2cb5b4c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:19.860778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 661/1331] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:20.157263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:20.157542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:20.157816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:20.159152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 671/1332] 10.222.0.49 () {54 vars in 964 bytes} [Mon Sep 26 05:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:20.164901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda1b6c-dac3-4a82-9c73-6c5828d4a76f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:20.165186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda1b6c-dac3-4a82-9c73-6c5828d4a76f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:20.165459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda1b6c-dac3-4a82-9c73-6c5828d4a76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:20.210937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:20.243305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-dbda1b6c-dac3-4a82-9c73-6c5828d4a76f None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:20.244472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-dbda1b6c-dac3-4a82-9c73-6c5828d4a76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:20.321634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 662/1333] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Sep 26 05:21:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:20.330743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:20.332306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:20.434641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88437c90-b218-46a3-b596-c8df72be0c6d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:20.506454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:20.506979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:20.507250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:20.507515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:20.509123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:20.535788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:20.617692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:20.619114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201185e-54ce-4b51-a36c-be84797055b1 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:20.639172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 672/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:23.067672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:23.068242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:23.068242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:23.069585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 663/1335] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:23.075680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203eacb-4d4f-4548-80cc-ec812a2d7dc2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:23.076108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203eacb-4d4f-4548-80cc-ec812a2d7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:23.076203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203eacb-4d4f-4548-80cc-ec812a2d7dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:23.141517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4203eacb-4d4f-4548-80cc-ec812a2d7dc2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:23.142725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4203eacb-4d4f-4548-80cc-ec812a2d7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:23.217287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:23.242521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 673/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:23.535889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:23.537439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:23.639767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577b2ab-7541-4355-a6c5-99ee41efd78e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:23.667330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:23.714491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:23.715149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:23.715417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:23.715681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:23.717285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:23.824616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:23.825960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0faf6-f3d2-4fef-87a7-36946eb52db6 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:23.843670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 664/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:26.663923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:26.664400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:26.664512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:26.665838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 674/1338] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:26.671306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f581-209f-4de0-8bca-9ca925824d85 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:26.671701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f581-209f-4de0-8bca-9ca925824d85 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:26.671855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f581-209f-4de0-8bca-9ca925824d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:26.737033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-92d3f581-209f-4de0-8bca-9ca925824d85 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:26.738237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-92d3f581-209f-4de0-8bca-9ca925824d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:26.746303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:26.836957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 665/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:27.117834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:27.119363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:27.221230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506e6cb-1268-4055-8d70-2268a260243e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:27.294836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:27.295433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:27.295711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:27.295982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:27.297603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:27.333023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:27.410188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:27.411570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ffa69-2408-42c5-aa0a-c43a56ad7a5a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:27.430447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 675/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:31.645290 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:31.645852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:31.645852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:31.647171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 666/1341] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:31.652647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757c1e-7827-491f-8232-958cbfffe263 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:31.653141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757c1e-7827-491f-8232-958cbfffe263 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:31.653237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757c1e-7827-491f-8232-958cbfffe263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:31.722871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-74757c1e-7827-491f-8232-958cbfffe263 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:31.724082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-74757c1e-7827-491f-8232-958cbfffe263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:31.816459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:31.828022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 676/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:32.111913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:32.113510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:32.216390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0c15-5199-4a34-8946-87e7ec62ae3a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:32.254063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:32.291146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:32.291757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:32.292036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:32.292334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:32.293927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:32.405121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:32.406457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2d90-81a5-4c34-882b-f55aca50cf95 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:32.424908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 667/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:35.223070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:35.223646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:35.223646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:35.224966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 677/1344] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:35.230809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e676b-91cc-4633-b0e5-d76587e79020 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:35.231341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e676b-91cc-4633-b0e5-d76587e79020 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:35.231454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e676b-91cc-4633-b0e5-d76587e79020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:35.298169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-980e676b-91cc-4633-b0e5-d76587e79020 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:35.299369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-980e676b-91cc-4633-b0e5-d76587e79020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:35.317623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:35.401088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 668/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:35.717685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:35.719242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:35.843014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3162a-db39-4334-9802-cbabbef427b0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:35.918514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:35.919114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:35.919395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:35.919660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:35.921262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:35.965018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:36.032545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:36.033982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34b735-71df-4238-9a88-114a876326b6 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:36.052770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 678/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:40.481826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:40.482361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:40.482361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:40.483958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 669/1347] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:40.489340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e7f81-569e-4d02-bfe6-5331d3316bbc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:40.489777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e7f81-569e-4d02-bfe6-5331d3316bbc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:40.489868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e7f81-569e-4d02-bfe6-5331d3316bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:40.559335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2a7e7f81-569e-4d02-bfe6-5331d3316bbc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:40.560566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2a7e7f81-569e-4d02-bfe6-5331d3316bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:40.655709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:40.658074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 679/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:40.937596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:40.939126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:41.039360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dab8b-db96-473d-8fea-0c54309246ec None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:41.085639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:41.113297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:41.113901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:41.114175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:41.114446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:41.116019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:41.224135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:41.225595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde52e2-d6ec-44f7-a9b3-cb4c360ca1b6 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:41.244183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 670/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:46.108337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6f84-97b1-4ab0-b0bf-48ccc52c777c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:46.108792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6f84-97b1-4ab0-b0bf-48ccc52c777c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:46.108876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6f84-97b1-4ab0-b0bf-48ccc52c777c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:46.110233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 680/1350] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:46.115316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc31f-37f3-4430-9ad9-9563563b4a49 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:46.115758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc31f-37f3-4430-9ad9-9563563b4a49 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:46.115995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abc31f-37f3-4430-9ad9-9563563b4a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:46.241491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b7abc31f-37f3-4430-9ad9-9563563b4a49 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:46.244007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b7abc31f-37f3-4430-9ad9-9563563b4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:46.297380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:46.398694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 671/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:50.462928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd9b74-362a-4ed2-ba3b-991485481b65 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:50.464537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd9b74-362a-4ed2-ba3b-991485481b65 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:50.465069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd9b74-362a-4ed2-ba3b-991485481b65 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:50.466530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 681/1352] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:21:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:50.471988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb7edb-10d0-4214-8cb7-3e8735978f08 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:50.472456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb7edb-10d0-4214-8cb7-3e8735978f08 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:50.472566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb7edb-10d0-4214-8cb7-3e8735978f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:50.555683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d4bb7edb-10d0-4214-8cb7-3e8735978f08 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:50.557011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d4bb7edb-10d0-4214-8cb7-3e8735978f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:50.634242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:50.674337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 672/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:21:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:52.224670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5c19-e3b4-486b-8325-e7c24ca158bc None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:52.225193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5c19-e3b4-486b-8325-e7c24ca158bc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:52.225334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5c19-e3b4-486b-8325-e7c24ca158bc None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:52.226707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 682/1354] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:52.231821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d81df-256d-42b2-b3df-deef05f97199 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:52.232226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d81df-256d-42b2-b3df-deef05f97199 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:52.232521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d81df-256d-42b2-b3df-deef05f97199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:52.310980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5d2d81df-256d-42b2-b3df-deef05f97199 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:52.312185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5d2d81df-256d-42b2-b3df-deef05f97199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:52.417379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 673/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:21:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:21:56.098779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb71fd-69ef-48a7-9a8b-5e0a46f35bf9 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:56.099276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb71fd-69ef-48a7-9a8b-5e0a46f35bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:56.099276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb71fd-69ef-48a7-9a8b-5e0a46f35bf9 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:56.100849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 683/1356] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:56.114586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5a6d-3717-46f7-9b8b-fbf76ddce540 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:56.115039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5a6d-3717-46f7-9b8b-fbf76ddce540 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:56.115127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5a6d-3717-46f7-9b8b-fbf76ddce540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:56.154194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:56.192992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-eb7b5a6d-3717-46f7-9b8b-fbf76ddce540 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:56.194516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-eb7b5a6d-3717-46f7-9b8b-fbf76ddce540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:56.316276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 674/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:56.615562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:56.616187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:56.616777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:56.618185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 684/1358] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:56.625814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c59235-c901-4cd1-aa26-6bf6bd9a79e8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:56.626100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c59235-c901-4cd1-aa26-6bf6bd9a79e8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:56.626378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c59235-c901-4cd1-aa26-6bf6bd9a79e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:56.719821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-01c59235-c901-4cd1-aa26-6bf6bd9a79e8 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:56.721067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-01c59235-c901-4cd1-aa26-6bf6bd9a79e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:56.771996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:56.845473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 675/1359] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:21:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:21:56.853067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:21:56.855146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:57.011846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae09805-2f3e-461f-b359-6052b9462c4a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:21:57.089649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:57.135190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:21:57.135860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:57.136092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:57.136386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:57.137971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:57.239193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:21:57.240879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680570-8206-44e7-ab0e-099048f7c403 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:21:57.263343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 685/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:21:59.681849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:59.682614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:59.682614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:59.684347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 676/1361] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:21:59.689751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e886e-ba01-47c5-82a5-10717f2a0dad None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:21:59.690343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e886e-ba01-47c5-82a5-10717f2a0dad None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:21:59.690343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e886e-ba01-47c5-82a5-10717f2a0dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:21:59.762536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a23e886e-ba01-47c5-82a5-10717f2a0dad None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:21:59.763457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a23e886e-ba01-47c5-82a5-10717f2a0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:21:59.808046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:21:59.872288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 686/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:00.157064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:00.158565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:00.334071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d3489-e884-4441-a7ed-fe7cc9e80d94 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:00.346776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:00.477204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:00.477876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:00.478389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:00.478753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:00.480431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:00.575080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:00.576709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1219f2-c7b9-49ff-9cd3-5ff3ab8d066c service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:00.594903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 677/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:03.097276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac730-f95c-4334-8573-44c5ce78406d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:03.097803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac730-f95c-4334-8573-44c5ce78406d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:03.097803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac730-f95c-4334-8573-44c5ce78406d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:03.099340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 687/1364] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:03.108157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14bbf-7eef-4048-935c-f7e94f543185 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:03.108157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14bbf-7eef-4048-935c-f7e94f543185 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:03.108157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14bbf-7eef-4048-935c-f7e94f543185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:03.122342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:03.176469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b4d14bbf-7eef-4048-935c-f7e94f543185 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:03.177830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b4d14bbf-7eef-4048-935c-f7e94f543185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:03.280781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 678/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:03.575184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:03.575468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:03.575752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:03.577230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 688/1366] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:03.582522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e363-bb8d-42de-b8a5-1af5620b0861 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:03.582983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e363-bb8d-42de-b8a5-1af5620b0861 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:03.583262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e363-bb8d-42de-b8a5-1af5620b0861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:03.671654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-adf3e363-bb8d-42de-b8a5-1af5620b0861 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:03.678172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-adf3e363-bb8d-42de-b8a5-1af5620b0861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:03.715730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:03.810567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 679/1367] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:22:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:03.818419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:03.819876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:03.975751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53b783-0ff1-49b2-8d29-94cd4b5781a8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:03.999107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:04.104580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:04.106168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:04.106672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:04.107166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:04.109310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:04.208321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:04.210087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e24ae2-3b39-4b84-9cb5-884a5493ecc4 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:04.230295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 689/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:04.374305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e5ce-ed3c-4ba0-997c-6e564c9d7c53 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:04.374842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e5ce-ed3c-4ba0-997c-6e564c9d7c53 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:04.374842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e5ce-ed3c-4ba0-997c-6e564c9d7c53 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:04.376385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 680/1369] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:04.382371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7932f0-ae24-4ab1-b1dc-9e3605fa66d8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:04.382667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7932f0-ae24-4ab1-b1dc-9e3605fa66d8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:04.382941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7932f0-ae24-4ab1-b1dc-9e3605fa66d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:04.423098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:04.467905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9d7932f0-ae24-4ab1-b1dc-9e3605fa66d8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:04.469290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d7932f0-ae24-4ab1-b1dc-9e3605fa66d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:04.578188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 690/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:04.910099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:04.910384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:04.910669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:04.912110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 681/1371] 10.222.0.49 () {54 vars in 963 bytes} [Mon Sep 26 05:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:04.916910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f014-f30e-4b41-abbb-58b841edfa5c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:04.917189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f014-f30e-4b41-abbb-58b841edfa5c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:04.917458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f014-f30e-4b41-abbb-58b841edfa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:04.998247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ea01f014-f30e-4b41-abbb-58b841edfa5c None None] MFA Rules not processed for user `0102a128d81741c88deeeafedbf0d390`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:04.999563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea01f014-f30e-4b41-abbb-58b841edfa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:05.047541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:05.084965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 691/1372] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Sep 26 05:22:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:05.088106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:05.089640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:05.202972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946185a-4456-4f53-83a4-7e81f840c23a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:05.253896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:05.302915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:05.303793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:05.304183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:05.304569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:05.307591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:05.477712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:05.480352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268c151-6823-455c-ae17-e29a5a4aad9e service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:05.514606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 682/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:06.494716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:06.495354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:06.495646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:06.497010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 692/1374] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:06.502327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df58c7-8f85-419e-a3a9-500e0e553122 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:06.502789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df58c7-8f85-419e-a3a9-500e0e553122 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:06.502881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df58c7-8f85-419e-a3a9-500e0e553122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:06.568583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a8df58c7-8f85-419e-a3a9-500e0e553122 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:06.569773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a8df58c7-8f85-419e-a3a9-500e0e553122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:06.590807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:06.667551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 683/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:06.955058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:06.956599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:07.125915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8322372-f60a-408a-a908-026c9e149d0f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:07.185061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:07.248797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:07.249429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:07.249702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:07.249970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:07.251592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:07.345957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:07.347630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941e6a-4b04-4ce0-951d-872af076cf0a service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:07.366274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 693/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:09.636995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:09.637543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:09.637843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:09.639414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 684/1377] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:09.644545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378f849-e257-4b1b-992b-8b9763eb6312 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:09.644970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378f849-e257-4b1b-992b-8b9763eb6312 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:09.645191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378f849-e257-4b1b-992b-8b9763eb6312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:09.720617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e378f849-e257-4b1b-992b-8b9763eb6312 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:09.721821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e378f849-e257-4b1b-992b-8b9763eb6312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:09.730273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:09.826867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 694/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:10.118281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:10.119936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:10.277278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2e831-cbf3-4980-8bd7-0903120374f8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:10.281384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:10.396499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:10.397269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:10.397650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:10.398026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:10.399756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:10.491466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:10.494213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac176431-2987-4a78-87eb-6404a68c1d1e service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:10.514149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 685/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:12.757075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:12.757663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:12.757663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:12.759654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 695/1380] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:12.764910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feab12e-95a3-49d7-8d80-86b6c810483e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:12.765335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feab12e-95a3-49d7-8d80-86b6c810483e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:12.765415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feab12e-95a3-49d7-8d80-86b6c810483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:12.776963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:12.836778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6feab12e-95a3-49d7-8d80-86b6c810483e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:12.838245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6feab12e-95a3-49d7-8d80-86b6c810483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:13.104757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 686/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:13.400046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:13.401590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:13.560942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:13.574271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ce91-9ee6-4219-9659-f4136f6d3778 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:13.692206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:13.692759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:13.693032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:13.693297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:13.694877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:13.792445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:13.794188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb474c-3eb6-463a-8209-d033616f52f3 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:13.804798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:13.814607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 696/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:16.047031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:16.047570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:16.047735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:16.049544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 687/1383] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:16.054705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ca62-027e-4ede-9893-a8b05051755a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:16.055144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ca62-027e-4ede-9893-a8b05051755a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:16.055241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ca62-027e-4ede-9893-a8b05051755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:16.130567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-04a3ca62-027e-4ede-9893-a8b05051755a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:16.132202 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-04a3ca62-027e-4ede-9893-a8b05051755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:16.248280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 697/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:16.535950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:16.537543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:16.632738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:16.697945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb777932-1cfa-454f-a22a-89f888e43b1a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:16.817656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:16.818177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:16.818462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:16.818730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:16.820339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:16.871749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:16.913678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:16.915541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b52079-3839-4018-b185-70f0ee5fc9e1 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:16.933217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 688/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:19.119940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:19.120481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:19.120641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:19.121856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 698/1386] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:19.127497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7425139-f07d-4fd5-a5ff-1232a4629e57 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:19.127943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7425139-f07d-4fd5-a5ff-1232a4629e57 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:19.128223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7425139-f07d-4fd5-a5ff-1232a4629e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:19.205829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b7425139-f07d-4fd5-a5ff-1232a4629e57 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:19.207304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b7425139-f07d-4fd5-a5ff-1232a4629e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:19.324470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 689/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:19.615675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:19.617335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:19.702016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:19.771665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969aa74-d568-40ec-ba42-656dbbe8985f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:19.895304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:19.895777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:19.896047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:19.896345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:19.897940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:19.937653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:19.989718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:19.991326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38598dc-93ca-461b-b43c-14454a2926ad service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:20.010272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 699/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:22.421537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:22.422147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:22.422147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:22.423552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 690/1389] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:22.428983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40de9-461e-4eef-b30e-2ac6f5a6da1f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:22.429468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40de9-461e-4eef-b30e-2ac6f5a6da1f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:22.429576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40de9-461e-4eef-b30e-2ac6f5a6da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:22.495381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-09d40de9-461e-4eef-b30e-2ac6f5a6da1f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:22.496604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-09d40de9-461e-4eef-b30e-2ac6f5a6da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:22.592361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 700/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:22.894403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:22.895903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:22.927354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:23.052093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f547ff3-1c0d-4b6e-a7d9-21419b91b217 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:23.156059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:23.168857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:23.169464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:23.169736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:23.170001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:23.171588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:23.265250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:23.266909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf190-2ca1-40d4-b6d8-21d48195b73d service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:23.285806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 691/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:22:23.686376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dca261-746d-4f3a-9268-5bb970e1286c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:23.686901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dca261-746d-4f3a-9268-5bb970e1286c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:23.686901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dca261-746d-4f3a-9268-5bb970e1286c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:23.688190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 701/1392] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:23.693472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290bd00-e0ad-4ee1-914e-4e617191d39c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:23.693952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290bd00-e0ad-4ee1-914e-4e617191d39c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:23.694039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290bd00-e0ad-4ee1-914e-4e617191d39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:23.761541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d290bd00-e0ad-4ee1-914e-4e617191d39c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:23.762733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d290bd00-e0ad-4ee1-914e-4e617191d39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:23.827142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:23.867083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 692/1393] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:24.169337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:24.169691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:24.170033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:24.171353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 702/1394] 10.222.0.49 () {54 vars in 962 bytes} [Mon Sep 26 05:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:24.176841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67894fe-251f-4bd3-ad24-ae82e4ff19fd None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:24.177127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67894fe-251f-4bd3-ad24-ae82e4ff19fd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:24.177400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67894fe-251f-4bd3-ad24-ae82e4ff19fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:24.259348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a67894fe-251f-4bd3-ad24-ae82e4ff19fd None None] MFA Rules not processed for user `0da5e4ab96bb4a9584682f32cbafb2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:24.260931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a67894fe-251f-4bd3-ad24-ae82e4ff19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:24.338033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 693/1395] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Sep 26 05:22:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:24.347259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:24.348817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:24.362975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:24.455516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6084b0-091b-4ba1-ae90-44aa1908dc07 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:24.531219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:24.531827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:24.532102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:24.532402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:24.533990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:24.603081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:24.645499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:24.646828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d15e0-66dc-4566-b51e-c9de2681651b service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:24.664579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 703/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:25.403988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c093efb-f1d6-44ef-bea1-4368750656fc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:25.404593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c093efb-f1d6-44ef-bea1-4368750656fc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:25.404704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c093efb-f1d6-44ef-bea1-4368750656fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:25.472405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3c093efb-f1d6-44ef-bea1-4368750656fc None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:25.473610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3c093efb-f1d6-44ef-bea1-4368750656fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:25.483558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:25.510236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481d7a-702c-4969-aed0-5e9a5083082a None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:25.510816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481d7a-702c-4969-aed0-5e9a5083082a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:25.510816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481d7a-702c-4969-aed0-5e9a5083082a None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:25.512260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 704/1397] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:25.517098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:25.517387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:25.517678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:25.565642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 694/1398] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:22:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:25.589534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:25.590827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:25.614729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1a08-2478-4319-9f67-9e168a570d88 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:25.615013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1a08-2478-4319-9f67-9e168a570d88 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:25.615289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1a08-2478-4319-9f67-9e168a570d88 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:25.616656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 695/1399] 10.222.0.49 () {54 vars in 943 bytes} [Mon Sep 26 05:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:22:25.623279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:25.623686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:25.624033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:25.692887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 705/1400] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:25.704023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] MFA Rules not processed for user `ed5f73bac406439e856d745885c0c092`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:25.705261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:25.796167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:25.818739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 696/1401] 10.222.0.49 () {58 vars in 1040 bytes} [Mon Sep 26 05:22:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:22:25.829002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:25.830197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:25.856466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:25.933963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8517c5-d604-4d20-9665-0e0fc550dced None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:25.987223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:25.988461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:26.040898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:26.041504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:26.041781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:26.042049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:26.043719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:26.085884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:26.087326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef022de6-3e1f-4838-9f1a-18368202c330 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:26.129022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:26.130628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c84d9f-7ff9-406e-9b1b-39af8c40fb05 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:26.139369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 706/1402] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:22:26.203151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:26.210542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:26.211148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:26.211419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:26.211684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:26.213967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:26.344943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:26.346593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917061-b575-409e-9167-63d33a84b4fb service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:26.367817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 697/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:22:28.613858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:28.614462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:28.614462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:28.615755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 707/1404] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:22:28.621926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2914d-7006-4978-b035-4ee88f532977 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:28.622372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2914d-7006-4978-b035-4ee88f532977 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:28.622461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2914d-7006-4978-b035-4ee88f532977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:28.693079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6ff2914d-7006-4978-b035-4ee88f532977 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:28.694337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6ff2914d-7006-4978-b035-4ee88f532977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:28.769217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:28.816417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 698/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:29.104574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:29.106506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:29.271749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f2f7d-f15c-4a36-a612-a4091c85b845 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:29.312810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:29.413024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:29.414431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:29.416175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:29.416905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:29.418704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:29.556552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:29.558416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d412bc2-22da-4c17-945d-92e47667b580 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:29.579801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 708/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:31.828516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:31.829385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:31.829385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:31.832369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 699/1407] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:31.836710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8022-e85f-40bd-abf8-14ac4bc9cf30 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:31.837825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8022-e85f-40bd-abf8-14ac4bc9cf30 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:31.837972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8022-e85f-40bd-abf8-14ac4bc9cf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:31.892553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:31.914741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-846a8022-e85f-40bd-abf8-14ac4bc9cf30 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:31.919327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-846a8022-e85f-40bd-abf8-14ac4bc9cf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:32.124141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 709/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:32.419351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:32.421112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:32.581852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa5e2a-2340-445c-9459-2d2f7faa87c5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:32.644131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:32.718457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:32.719196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:32.719544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:32.719820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:32.721494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:32.822403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:32.824008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0b717-0936-46c8-baea-3288be16bf85 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:32.842857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 700/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:42.122643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=47 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:42.840536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=75 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:49.335557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6914868-432a-418e-9452-fc273bc5db4f None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:49.336097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6914868-432a-418e-9452-fc273bc5db4f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:49.336097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6914868-432a-418e-9452-fc273bc5db4f None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:49.337667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 710/1410] 10.222.0.49 () {54 vars in 918 bytes} [Mon Sep 26 05:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:49.342866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087fea-9dfb-4278-b5f9-e3318d92145d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:49.343267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087fea-9dfb-4278-b5f9-e3318d92145d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:49.343400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087fea-9dfb-4278-b5f9-e3318d92145d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:49.418579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-61087fea-9dfb-4278-b5f9-e3318d92145d None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:49.419709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-61087fea-9dfb-4278-b5f9-e3318d92145d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:49.524201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 701/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Sep 26 05:22:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:49.538493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd85b5-343c-4771-9125-198217f669b5 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:49.538778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd85b5-343c-4771-9125-198217f669b5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:49.539051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd85b5-343c-4771-9125-198217f669b5 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:49.540423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 711/1412] 10.222.0.49 () {54 vars in 918 bytes} [Mon Sep 26 05:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:49.545384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c4e5-7b3e-42dc-9670-0f9d83f7a9ea None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:49.545672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c4e5-7b3e-42dc-9670-0f9d83f7a9ea None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:49.545946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c4e5-7b3e-42dc-9670-0f9d83f7a9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:49.620336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:49.623404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9531c4e5-7b3e-42dc-9670-0f9d83f7a9ea None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:49.625181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9531c4e5-7b3e-42dc-9670-0f9d83f7a9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:49.739580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 702/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Sep 26 05:22:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:52.328059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:52.328656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:52.328656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:52.329941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:52.335124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c0f8f-5adf-4dd0-b41d-f63167cf67dc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:52.335584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c0f8f-5adf-4dd0-b41d-f63167cf67dc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:52.335712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c0f8f-5adf-4dd0-b41d-f63167cf67dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:52.401866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-db2c0f8f-5adf-4dd0-b41d-f63167cf67dc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:52.403065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-db2c0f8f-5adf-4dd0-b41d-f63167cf67dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:52.459869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:52.501654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:52.795035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:52.796608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:52.939446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80514452-543c-4c65-98f3-6fffbae09500 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:53.013801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:53.054513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:53.055030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:53.055307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:53.055576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:53.057729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:53.150192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:53.151795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b43f-7bd9-4d81-a8cd-336ac54cd83f service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:53.169416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 713/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:54.326271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:54.326848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:54.326848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:54.328163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:54.334348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801d-b51f-43f8-949a-4fecdca8488e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:54.334826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801d-b51f-43f8-949a-4fecdca8488e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:54.334936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801d-b51f-43f8-949a-4fecdca8488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:54.411716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3aec801d-b51f-43f8-949a-4fecdca8488e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:54.412920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3aec801d-b51f-43f8-949a-4fecdca8488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:54.440485 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:54.511380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:54.677137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:54.678554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:54.827608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6092-e06f-4d50-a9ec-b8f03af5404e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:54.867846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:54.941590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:54.942179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:54.942460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:54.942727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:54.944343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:55.032462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:55.034034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462b9e-47f3-4d0d-b32f-c160fba0ee9e service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:55.052414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 705/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:56.183041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:56.183598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:56.183598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:56.184999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:56.191065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c286-c6f6-4ccd-b52e-83e32c156194 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:56.191462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c286-c6f6-4ccd-b52e-83e32c156194 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:56.191564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c286-c6f6-4ccd-b52e-83e32c156194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:56.246644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:56.270445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c521c286-c6f6-4ccd-b52e-83e32c156194 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:56.271638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c521c286-c6f6-4ccd-b52e-83e32c156194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:56.366609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:56.518214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:56.519756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:56.664644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25338615-97d6-4628-8310-0e35b47a7e56 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:56.677551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:56.778515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:56.779045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:56.779322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:56.779593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:56.781235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:56.870227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:56.871818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dab3-e41b-42b8-86dd-25613d27cc52 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:56.889426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 716/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:57.998634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:57.999190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:57.999190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:58.000546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:58.006737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c766-c3de-49d7-b7b4-77b7129b4a2c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:58.007160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c766-c3de-49d7-b7b4-77b7129b4a2c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:58.007281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c766-c3de-49d7-b7b4-77b7129b4a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:58.027597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:58.082755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3396c766-c3de-49d7-b7b4-77b7129b4a2c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:58.083864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3396c766-c3de-49d7-b7b4-77b7129b4a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:58.178384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:22:58.333662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:22:58.335101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:58.447362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:58.484729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce12ed-f886-40d5-8c83-e3947c0a5867 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:22:58.596129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:22:58.596753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:58.597032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:58.597305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:58.598888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:58.674956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:22:58.689366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:22:58.690982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fe324-2dfc-4005-be37-50c03e3031ac service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:22:58.709205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 708/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:22:59.815922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:59.816528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:59.816528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:59.817815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:22:59.824065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf181e8-363a-44e4-ac65-a1506debb3b9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:22:59.824543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf181e8-363a-44e4-ac65-a1506debb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:22:59.824651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf181e8-363a-44e4-ac65-a1506debb3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:22:59.900272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9bf181e8-363a-44e4-ac65-a1506debb3b9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:22:59.901413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9bf181e8-363a-44e4-ac65-a1506debb3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:22:59.999435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:00.155954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:00.157439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:00.241806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:00.303376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593946-a9ed-4028-b052-7a69fe7d72fe None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:00.413715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:00.414244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:00.414521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:00.414790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:00.416455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:00.465993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:00.506731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:00.508344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35bd67-db9a-4587-8177-88e27563f527 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:00.526701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 719/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:01.590655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:01.591236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:01.591236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:01.592579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:01.598606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049709aa-8145-43a7-ba24-7b68bd0b465e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:01.599020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049709aa-8145-43a7-ba24-7b68bd0b465e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:01.599143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049709aa-8145-43a7-ba24-7b68bd0b465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:01.674310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-049709aa-8145-43a7-ba24-7b68bd0b465e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:01.675496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-049709aa-8145-43a7-ba24-7b68bd0b465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:01.772150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:01.922267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:01.923761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:01.982045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:02.080729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a38a4d-60c2-4d9c-a2cb-493317bf393d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:02.193365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:02.193961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:02.194233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:02.194495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:02.196048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:02.202947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:02.284129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:02.285739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4c1c5-1f4f-455f-a3f8-275f987974b7 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:02.303687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 711/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:03.344036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:03.344604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:03.344604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:03.345930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:03.352145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7920e-eac2-47e8-a5a7-037a7f582697 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:03.352604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7920e-eac2-47e8-a5a7-037a7f582697 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:03.352721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7920e-eac2-47e8-a5a7-037a7f582697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:03.428606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c5e7920e-eac2-47e8-a5a7-037a7f582697 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:03.429804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c5e7920e-eac2-47e8-a5a7-037a7f582697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:03.498578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:03.528881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:03.679424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:03.680906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:03.699095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:03.827513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2752c3-0511-46db-885f-6732805a5f96 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:03.912211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:03.939109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:03.939648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:03.939930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:03.940200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:03.941833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:04.032206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:04.033832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e48249-4621-42b5-8866-a0f2f625cce6 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:04.051775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 722/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:05.131732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:05.132217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:05.132344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:05.133654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:05.140395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bcda-f16b-4484-a295-411ce3de4b77 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:05.140854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bcda-f16b-4484-a295-411ce3de4b77 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:05.140971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bcda-f16b-4484-a295-411ce3de4b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:05.214770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c719bcda-f16b-4484-a295-411ce3de4b77 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:05.215984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c719bcda-f16b-4484-a295-411ce3de4b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:05.257664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:05.314048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:05.467778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:05.469318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:05.615433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0546fc-b7a9-4cb1-b343-d53d2d7980ca None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:05.669239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:05.729105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:05.729623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:05.729881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:05.730149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:05.731730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:05.821159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:05.823391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c9400-1017-46c9-9c65-2426bd781cd5 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:05.841253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 714/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:06.896946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:06.897364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:06.897517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:06.898865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:23:06.905933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128977d9-67da-4e32-8379-af9feafe1e1b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:06.906370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128977d9-67da-4e32-8379-af9feafe1e1b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:06.906476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128977d9-67da-4e32-8379-af9feafe1e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:06.982952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-128977d9-67da-4e32-8379-af9feafe1e1b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:06.984155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-128977d9-67da-4e32-8379-af9feafe1e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:06.986481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:07.086466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:07.236531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:07.238108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:07.387234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9364f15-bacd-48db-963d-ea8607f06a20 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:07.413598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:07.500381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:07.500914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:07.501211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:07.501503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:07.503078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:07.589259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:07.590854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4feac2e-0970-42ca-8861-70c15ad97f96 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:07.608948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 725/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:09.672349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:09.672931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:09.672931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:09.674298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:09.679877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88164f7-fce5-42f2-8fa3-c3fa9598d36e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:09.680409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88164f7-fce5-42f2-8fa3-c3fa9598d36e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:09.680505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88164f7-fce5-42f2-8fa3-c3fa9598d36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:09.722678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:09.748769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c88164f7-fce5-42f2-8fa3-c3fa9598d36e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:09.749973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c88164f7-fce5-42f2-8fa3-c3fa9598d36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:09.851336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:10.141471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:10.143015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:10.245531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840122d-bbb5-42b1-ad41-cbc8e2e56b7f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:10.278028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:10.319495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:10.320094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:10.320405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:10.320682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:10.322285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:10.429241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:10.430661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657545bf-0ad2-4960-bd58-8f20e24b307f service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:10.449021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 717/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:12.744879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:12.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:12.745453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:12.746769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:12.752159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b024d-cea4-47ba-8c93-97ff1929bfc5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:12.752619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b024d-cea4-47ba-8c93-97ff1929bfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:12.752744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b024d-cea4-47ba-8c93-97ff1929bfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:12.820893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ab2b024d-cea4-47ba-8c93-97ff1929bfc5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:12.822785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ab2b024d-cea4-47ba-8c93-97ff1929bfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:12.835284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:12.918623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:13.206508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:13.208105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:13.318240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b270b-706a-4b5d-b693-2ab8ce7872a6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:13.331858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:13.394204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:13.394700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:13.394980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:13.395236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:13.396836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:13.508309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:13.510523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01596c06-15e3-4e72-b97d-766d1a2ce982 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:13.528718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 728/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:15.998752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:15.999284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:15.999284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:16.000674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:16.005932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b701-89c5-41f4-9eaf-0cf92daf07e5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:16.006345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b701-89c5-41f4-9eaf-0cf92daf07e5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:16.006498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5b701-89c5-41f4-9eaf-0cf92daf07e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:16.054200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:16.072901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-78f5b701-89c5-41f4-9eaf-0cf92daf07e5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:16.074116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-78f5b701-89c5-41f4-9eaf-0cf92daf07e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:16.174476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:23:16.464208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:16.465766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:16.569707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2fd41-9a53-48d7-b437-bebdfde92eff None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:16.610166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:16.644619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:16.645220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:16.645495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:16.645762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:16.647323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:16.755005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:16.756362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdc47f-08c2-4cf6-beda-1caa9667a35b service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:16.774451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 720/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:19.112602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:19.113032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:19.113141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:19.114477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:19.120373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3e99-3d45-43bc-bbbb-3e28c7cb8e71 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:19.120814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3e99-3d45-43bc-bbbb-3e28c7cb8e71 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:19.120913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3e99-3d45-43bc-bbbb-3e28c7cb8e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:19.187228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-88cc3e99-3d45-43bc-bbbb-3e28c7cb8e71 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:19.188448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-88cc3e99-3d45-43bc-bbbb-3e28c7cb8e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:19.209310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:19.285382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:19.569446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:19.570991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:19.673217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82c094-3e9a-4faa-8fd4-4dacc2ca81b9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:19.693702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:19.747232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:19.747833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:19.748111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:19.748480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:19.750074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:19.861775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:19.863165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c772ed6-6868-47a8-8e72-ee153b90eb30 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:19.880963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 731/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:22.088517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:22.089136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:22.089136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:22.090389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:22.095792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2f3e4-828f-44bb-a3b3-f4d1022faafe None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:22.096221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2f3e4-828f-44bb-a3b3-f4d1022faafe None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:22.096385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2f3e4-828f-44bb-a3b3-f4d1022faafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:22.161015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:22.163760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a5f2f3e4-828f-44bb-a3b3-f4d1022faafe None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:22.166227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5f2f3e4-828f-44bb-a3b3-f4d1022faafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:22.262168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:22.577269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:22.578794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:22.681802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd7094-df98-4d14-abe5-8b6cdff1a89f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:22.853353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:22.879201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:22.879817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:22.880091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:22.880388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:22.881967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:22.990625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:22.992049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d267d-8289-4fc0-8e0f-f7afc57fb325 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:23.010838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 723/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:25.243189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:25.244288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:25.244288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:25.245751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:25.250805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a515e-f726-4220-89b9-c700428db36c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:25.251241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a515e-f726-4220-89b9-c700428db36c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:25.251384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a515e-f726-4220-89b9-c700428db36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:25.317562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6e6a515e-f726-4220-89b9-c700428db36c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:25.318757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6e6a515e-f726-4220-89b9-c700428db36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:25.346318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:25.413281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:25.704097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:25.705712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:25.814281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a07e8d-c305-4058-9e1e-622e144ee399 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:25.843047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:25.888742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:25.889356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:25.889638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:25.889910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:25.891507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:26.002050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:26.003496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3dcdd-1dff-494c-8016-719e19f2fa77 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:26.021897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 734/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:28.279029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113609e-36ca-4320-a481-90cccf404496 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:28.279547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113609e-36ca-4320-a481-90cccf404496 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:28.279673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113609e-36ca-4320-a481-90cccf404496 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:28.280966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:28.286390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022af6-31c2-4c0c-ada6-0caf678f7a47 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:28.286823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022af6-31c2-4c0c-ada6-0caf678f7a47 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:28.286922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022af6-31c2-4c0c-ada6-0caf678f7a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:28.352851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-44022af6-31c2-4c0c-ada6-0caf678f7a47 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:28.354557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-44022af6-31c2-4c0c-ada6-0caf678f7a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:28.363029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:28.453866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:28.741073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113609e-36ca-4320-a481-90cccf404496 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:28.742653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7113609e-36ca-4320-a481-90cccf404496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:28.846464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113609e-36ca-4320-a481-90cccf404496 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:28.903576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:28.921963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:28.922555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:28.922827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:28.923093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:28.924718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:29.031732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:29.033142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10c6d8-0624-4086-9e0b-9ecd92f47176 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:29.051351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 726/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:31.238514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:31.239141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:31.239141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:31.240405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:31.245947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952959a3-c61f-452e-8de2-12a4dc8508d0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:31.246377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952959a3-c61f-452e-8de2-12a4dc8508d0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:31.246496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952959a3-c61f-452e-8de2-12a4dc8508d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:31.311438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-952959a3-c61f-452e-8de2-12a4dc8508d0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:31.312640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-952959a3-c61f-452e-8de2-12a4dc8508d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:31.350717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:31.408426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:31.687542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:31.689197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:31.792575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de476fd-9fee-444c-acb4-ba4a2c6bfe84 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:31.830912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:31.870082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:31.870685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:31.870960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:31.871227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:31.872850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:31.984933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:31.986622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6feca-572b-4527-ab80-305a8267923a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:32.005227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 737/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:34.176275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:34.176836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:34.176836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:34.178160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:34.183669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ce3f-93c1-4266-b4c1-66711bb759e3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:34.184108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ce3f-93c1-4266-b4c1-66711bb759e3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:34.184220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ce3f-93c1-4266-b4c1-66711bb759e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:34.250782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-48e7ce3f-93c1-4266-b4c1-66711bb759e3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:34.251880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-48e7ce3f-93c1-4266-b4c1-66711bb759e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:34.269540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:34.349000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:34.629258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:34.630772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:34.733077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f062d5a-f887-4c25-8ff2-d68486f1701c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:34.801221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:34.808410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:34.809006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:34.809286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:34.809582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:34.811157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:34.918328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:34.919760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dc7fb-b3f0-4c43-b2d5-8744763ce078 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:34.938462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 729/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:37.200467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:37.201039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:37.201039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:37.202367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:37.207833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc847c1-d79d-4ec2-82aa-f25b6980c115 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:37.208280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc847c1-d79d-4ec2-82aa-f25b6980c115 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:37.208422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc847c1-d79d-4ec2-82aa-f25b6980c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:37.274580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0dc847c1-d79d-4ec2-82aa-f25b6980c115 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:37.276380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0dc847c1-d79d-4ec2-82aa-f25b6980c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:37.324904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:37.373294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:37.660717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:37.662255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:37.764117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417cd7-c83d-4d41-a629-dd83371a24f9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:37.813458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:37.839560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:37.840148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:37.840453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:37.840720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:37.842310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:37.950484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:37.951953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a93a-55ba-4036-aa21-35d7aae03211 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:37.970460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 740/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:23:47.371755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=22 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:47.968126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=66 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:58.044374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dda6c-808b-4bc1-8e6d-e3a3f8d308d6 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:58.044887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dda6c-808b-4bc1-8e6d-e3a3f8d308d6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:58.044887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dda6c-808b-4bc1-8e6d-e3a3f8d308d6 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:58.046206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 731/1471] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:58.051326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca0ef-6b35-430c-b212-bc856c3e0d63 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:58.051761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca0ef-6b35-430c-b212-bc856c3e0d63 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:58.051907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca0ef-6b35-430c-b212-bc856c3e0d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:58.119631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-477ca0ef-6b35-430c-b212-bc856c3e0d63 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:58.120849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-477ca0ef-6b35-430c-b212-bc856c3e0d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:58.216529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 741/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:58.517739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe738-d578-4447-8985-da19416fbb6f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:58.518027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe738-d578-4447-8985-da19416fbb6f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:58.518303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe738-d578-4447-8985-da19416fbb6f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:58.519636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 732/1473] 10.222.0.49 () {54 vars in 962 bytes} [Mon Sep 26 05:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:23:58.525357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779932-fcf2-4f60-8ce5-095c7bdbb77a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:58.525665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779932-fcf2-4f60-8ce5-095c7bdbb77a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:58.526016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779932-fcf2-4f60-8ce5-095c7bdbb77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:58.600577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-05779932-fcf2-4f60-8ce5-095c7bdbb77a None None] MFA Rules not processed for user `0da5e4ab96bb4a9584682f32cbafb2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:23:58.601771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-05779932-fcf2-4f60-8ce5-095c7bdbb77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:58.622528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:58.677660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 742/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Sep 26 05:23:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:23:58.686085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe738-d578-4447-8985-da19416fbb6f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:23:58.687579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-127fe738-d578-4447-8985-da19416fbb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:58.790696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe738-d578-4447-8985-da19416fbb6f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:23:58.862455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:23:58.863049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:23:58.863322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:23:58.863589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:23:58.865179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:23:58.919277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:23:58.973417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:23:58.974828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380a165-1652-4cf9-b54e-a3658c6a67c6 service glance] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:23:58.993310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 733/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:24:01.234960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:01.235531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:01.235531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:01.237138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 743/1476] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:01.242497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3e33-099c-437b-98ba-71062d498896 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:01.242916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3e33-099c-437b-98ba-71062d498896 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:01.243026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3e33-099c-437b-98ba-71062d498896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:01.308669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0c9c3e33-099c-437b-98ba-71062d498896 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:01.310001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0c9c3e33-099c-437b-98ba-71062d498896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:01.405848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 734/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:24:01.680780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:01.682760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:01.807002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbb397-14fd-4855-9d95-fb26b0ea2188 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:01.856676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:01.902464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:01.903068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:01.903343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:01.903610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:01.904236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:01.906047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:24:01.906047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:24:01.906047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:24:01.907189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8921d90-14c8-4d45-8d55-58fbb5307ff3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:01.926359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 744/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 05:24:01 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:24:04.102611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:04.103197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:04.103481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:04.105065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 735/1479] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:04.110625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715aa7-79a6-4c9e-be49-0165207ef121 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:04.111019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715aa7-79a6-4c9e-be49-0165207ef121 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:04.111441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715aa7-79a6-4c9e-be49-0165207ef121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:04.176335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d1715aa7-79a6-4c9e-be49-0165207ef121 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:04.177520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d1715aa7-79a6-4c9e-be49-0165207ef121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:04.274354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 745/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:24:04.570932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:04.572672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:04.582613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:04.717243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6e43f-a7b6-42a9-9c8e-c62231e264be None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:04.796517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:04.829257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:04.829805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:04.830057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:04.830318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:04.831882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:04.921350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:04.922913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f1957-d6b0-406a-b07c-57c96b45c669 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:04.940547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 736/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:24:07.247363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:07.247924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:07.247924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:07.249259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 746/1482] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:07.254426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490cf0-a16b-4c17-8077-4594d815ed25 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:07.254882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490cf0-a16b-4c17-8077-4594d815ed25 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:07.254990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490cf0-a16b-4c17-8077-4594d815ed25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:07.320776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-93490cf0-a16b-4c17-8077-4594d815ed25 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:07.321990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-93490cf0-a16b-4c17-8077-4594d815ed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:07.370422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:07.420547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 737/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:24:07.704611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:07.706156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:07.730444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:07.812476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b01b1a-3d23-4c04-b78b-765495455c5b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:07.885289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:07.885888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:07.886158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:07.886423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:07.888012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:07.961944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:07.998025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:07.999461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58429fb-6cdf-4272-bbd1-a85f9dda4de4 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:08.017746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 747/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:24:10.265828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:10.266360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:10.266360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:10.267650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 738/1485] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:10.272994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2f3a-e52f-483d-803b-3dab9eb14300 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:10.273451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2f3a-e52f-483d-803b-3dab9eb14300 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:10.273578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2f3a-e52f-483d-803b-3dab9eb14300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:10.349216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-045c2f3a-e52f-483d-803b-3dab9eb14300 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:10.350436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-045c2f3a-e52f-483d-803b-3dab9eb14300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:10.460258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 748/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:24:10.747988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:10.749551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:10.851603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970870-5f41-42c2-a6a1-2ac620e70f3a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:10.922906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:10.924533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:10.925168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:10.925644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:10.926080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:10.927911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:11.058612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:11.060415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05537000-6241-43b2-8678-c8fe5a5a7f1c service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:11.080169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 739/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:24:13.393291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:13.393850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:13.393850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:13.395097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 749/1488] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:13.400579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a75d-c92e-452d-8573-b571a678b2ea None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:13.401035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a75d-c92e-452d-8573-b571a678b2ea None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:13.401137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a75d-c92e-452d-8573-b571a678b2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:13.470931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e431a75d-c92e-452d-8573-b571a678b2ea None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:13.472120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e431a75d-c92e-452d-8573-b571a678b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:13.526374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:13.576289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 740/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:24:13.863661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:13.865243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:13.901171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:13.968133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075499f-baae-42dc-9cea-f912232fdd47 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:14.041486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:14.042089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:14.042365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:14.042639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:14.044237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:14.241536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:14.271598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:14.273062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1350d-9b54-4fad-a57f-2154609ce77d service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:14.291344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 750/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:24:16.735081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:16.735625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:16.735625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:16.737279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 741/1491] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 05:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:24:16.742501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00f64-a8b1-4147-81b1-5356e2956c6b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:16.742951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00f64-a8b1-4147-81b1-5356e2956c6b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:16.743049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00f64-a8b1-4147-81b1-5356e2956c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:16.809364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b5b00f64-a8b1-4147-81b1-5356e2956c6b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:24:16.810599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b5b00f64-a8b1-4147-81b1-5356e2956c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:16.906415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 751/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 05:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:24:17.200561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:24:17.202077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:17.304597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ff4c8-fc15-4f7c-8158-6cb4f7aebaab None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:24:17.378073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:24:17.378597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:24:17.378875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:24:17.379143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:24:17.380832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:24:17.389478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:17.492114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:24:17.493596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfc334-cac4-414d-8b2d-0ddfb24d6966 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:24:17.511714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 742/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:24:26.904938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=82 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:24:27.509336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=50 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:36.482847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:36.483465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:36.483465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:36.557799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:25:36.558986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:36.653374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 752/1494] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:25:36.661497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:36.662011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:36.662119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:36.737832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:25:36.738961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:36.834829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 743/1495] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:25:36.840307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:36.841504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:36.852773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f351-7edd-4e0f-ad84-f8703134d057 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:36.914123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:36.947472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:36.948078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:36.948384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:36.948657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:36.949291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:25:36.949590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:36.950617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:36.950617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:25:36.950617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:25:36.951495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:36.963249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 753/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:25:36.971513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:36.980579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2f5a1-2782-4bcf-8cfb-d4090bb5d7b7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:37.041563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:37.074891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:37.075502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.075756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.076030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.076688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:37.078401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:37.078401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:25:37.078401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:25:37.079569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:37.136216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 744/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:25:36 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:25:37.140403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:37.141907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:37.174167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa622-4172-4599-ad78-04984c6e9783 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:37.266945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:37.267547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.267818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.268082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.268730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:37.269767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:37.269767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:25:37.269767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:25:37.270948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:37.313870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 754/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:25:37 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:25:37.317950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:37.319467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:37.380164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f065-3f5a-47eb-b963-19e35d84c898 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:37.462307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:37.473446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:37.474043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.474319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.474587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.475220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:37.477034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:37.477034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:25:37.477034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:25:37.477914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:37.490492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 745/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:25:37.495081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:37.496614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:37.507877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82695727-ef45-4f81-b76d-bf89ea75a608 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:37.532083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:37.602474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:37.603001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.603273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.603539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.604163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:37.605190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:37.605190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:25:37.605190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:25:37.606073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fcf49-32f7-4865-822f-7b2eee5b172e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:37.618972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 755/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:25:37.623145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:37.624730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:37.657400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ce1d-ef69-41c1-a0e0-71f0ec5cb7c3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:37.750578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:37.751296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.751494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.751765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] PATH_INFO: `/v3/projects/3a79f6d2ba814d72bc56ec76600697ac/users/925201a88d174f3bbcc1f51c8cee73f3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.774733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a79f6d2ba814d72bc56ec76600697ac, user_id=925201a88d174f3bbcc1f51c8cee73f3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:37.775788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:37.775788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:25:37.775788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:25:37.777734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5c216-33ea-4f5a-b5e4-401f6518837e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:37.806217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 746/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:25:37 2022] PUT /identity/v3/projects/3a79f6d2ba814d72bc56ec76600697ac/users/925201a88d174f3bbcc1f51c8cee73f3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:25:37.818504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae571a06-dee1-4b30-add5-3fbd41fab638 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.818790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae571a06-dee1-4b30-add5-3fbd41fab638 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.819064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae571a06-dee1-4b30-add5-3fbd41fab638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.876387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ae571a06-dee1-4b30-add5-3fbd41fab638 None None] MFA Rules not processed for user `925201a88d174f3bbcc1f51c8cee73f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:25:37.877506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ae571a06-dee1-4b30-add5-3fbd41fab638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:37.954420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 756/1502] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:25:37.968057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.968367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.968645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:37.969972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 747/1503] 10.222.0.49 () {54 vars in 963 bytes} [Mon Sep 26 05:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:25:37.974505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41598de4-b84c-489b-81ee-3523b1e54793 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:37.974787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41598de4-b84c-489b-81ee-3523b1e54793 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:37.975060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41598de4-b84c-489b-81ee-3523b1e54793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:38.006972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:38.052746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-41598de4-b84c-489b-81ee-3523b1e54793 None None] MFA Rules not processed for user `0102a128d81741c88deeeafedbf0d390`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:25:38.053939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-41598de4-b84c-489b-81ee-3523b1e54793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:38.131413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 757/1504] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Sep 26 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:25:38.139543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:38.141121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:38.190415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:38.244594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d3210-4129-495d-bec8-5b184b941c8c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:38.316933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:38.317529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:38.317804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:38.318070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:38.319643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:38.422942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:38.424417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:38.427812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:38.443221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 748/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:25:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:25:38.489496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620766-f9c9-4212-a2df-8c33d136be04 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:38.489782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620766-f9c9-4212-a2df-8c33d136be04 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:38.490053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620766-f9c9-4212-a2df-8c33d136be04 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:38.491299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 758/1506] 10.222.0.49 () {52 vars in 830 bytes} [Mon Sep 26 05:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:25:38.495429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:38.496661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:38.594539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b907be-8c5a-4560-be6c-0fad375097c1 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:38.687352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:38.687899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:38.688173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:38.688472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] PATH_INFO: `/v3/users/925201a88d174f3bbcc1f51c8cee73f3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:38.701795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] RBAC: Authorizing `identity:delete_user(user_id=925201a88d174f3bbcc1f51c8cee73f3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:38.702803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:38.702803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:25:38.702803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:25:38.703989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:38.713569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:38.769972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925201a88d174f3bbcc1f51c8cee73f3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:38.779756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925201a88d174f3bbcc1f51c8cee73f3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:38.780409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925201a88d174f3bbcc1f51c8cee73f3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:38.795709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925201a88d174f3bbcc1f51c8cee73f3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:38.796193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925201a88d174f3bbcc1f51c8cee73f3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:38.824089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 749/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:25:38 2022] DELETE /identity/v3/users/925201a88d174f3bbcc1f51c8cee73f3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:25:38.828723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:38.829203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:38.829302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:38.889814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:25:38.890995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:38.967077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:38.979602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 759/1508] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:25:38.989413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:38.990622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:39.092912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07adb4-24f6-4ffe-8b24-39d9680b628c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:39.165208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:39.186380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:39.186993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:39.187274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:39.187548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:39.189185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:39.298170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:39.299516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142356e-9de9-4112-821d-74ed252fefa7 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:39.317877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 750/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:25:39.642530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:25:39.643744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:25:39.655280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec21839-6e4a-4db8-ac99-c4ec6a153679 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:25:39.753936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:25:39.754560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:25:39.754831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:25:39.755097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] PATH_INFO: `/v3/projects/3a79f6d2ba814d72bc56ec76600697ac` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:25:39.764974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a79f6d2ba814d72bc56ec76600697ac)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:25:39.765999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:25:39.765999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:25:39.765999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:25:39.767183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:25:39.827823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a79f6d2ba814d72bc56ec76600697ac', 'request_id': 'req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:39.838332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a79f6d2ba814d72bc56ec76600697ac', 'request_id': 'req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:39.844458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a79f6d2ba814d72bc56ec76600697ac', 'request_id': 'req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:39.875628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] The token cache is being invalidate because project 3a79f6d2ba814d72bc56ec76600697ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:25:39.875916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:39.876480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c7c57d4-0937-40b7-bf6d-53c0a207fa51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:25:39.877782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 760/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:25:39 2022] DELETE /identity/v3/projects/3a79f6d2ba814d72bc56ec76600697ac => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:25:49.315607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=62 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:25:49.873639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:27:08.445306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:08.445807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:08.445914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:08.447182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:08.451931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c391-0c4d-4ae6-966f-26028ed71a51 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:08.452409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c391-0c4d-4ae6-966f-26028ed71a51 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:08.452517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c391-0c4d-4ae6-966f-26028ed71a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:08.531225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e1f2c391-0c4d-4ae6-966f-26028ed71a51 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:08.532431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e1f2c391-0c4d-4ae6-966f-26028ed71a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:08.632841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:08.784935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:08.788828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:08.906940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73769-5893-4d0c-8443-232b481ff263 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:08.912373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:08.913584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:08.913861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:08.914135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:08.922190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:08.923791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca36215-cd09-4569-be04-a0e82e712335 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:08.933474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 752/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:10.008051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82bcbf-c7ef-482d-a271-427290daa39c None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:10.008692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82bcbf-c7ef-482d-a271-427290daa39c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:10.008692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82bcbf-c7ef-482d-a271-427290daa39c None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:10.009991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:10.016134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b21a2-bfcd-465d-846b-938e625a57d2 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:10.016585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b21a2-bfcd-465d-846b-938e625a57d2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:10.016706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b21a2-bfcd-465d-846b-938e625a57d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:10.037368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-de1b21a2-bfcd-465d-846b-938e625a57d2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:10.038633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-de1b21a2-bfcd-465d-846b-938e625a57d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:10.046702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:10.200026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82bcbf-c7ef-482d-a271-427290daa39c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:10.215063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82bcbf-c7ef-482d-a271-427290daa39c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:10.220394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:10.220985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:10.221257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:10.221523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:10.229576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:10.231141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057feda-16e3-48ab-be03-5fc3d6e1f646 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:10.240591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 763/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:12.842011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1734-d28a-4c13-a4d0-fb79a3a6b2f9 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:12.842524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1734-d28a-4c13-a4d0-fb79a3a6b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:12.842524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea1734-d28a-4c13-a4d0-fb79a3a6b2f9 None None] PATH_INFO: `/v3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:12.843738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 754/1517] 10.222.0.60 () {54 vars in 925 bytes} [Mon Sep 26 05:27:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:27:12.854178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b30d5a-ee25-453d-9084-9911acf82397 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:12.854601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b30d5a-ee25-453d-9084-9911acf82397 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:12.854804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b30d5a-ee25-453d-9084-9911acf82397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:12.912915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-97b30d5a-ee25-453d-9084-9911acf82397 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:12.914093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-97b30d5a-ee25-453d-9084-9911acf82397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:12.930180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:27:12.984825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 764/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Sep 26 05:27:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:17.347699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6ed8c-e4bb-401a-989d-7b2ee20da054 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:17.348195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6ed8c-e4bb-401a-989d-7b2ee20da054 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:17.348362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6ed8c-e4bb-401a-989d-7b2ee20da054 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:17.349650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 755/1519] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:17.354622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf38d3-c6ce-49a6-b4fa-ac5d733d2947 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:17.355087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf38d3-c6ce-49a6-b4fa-ac5d733d2947 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:17.355196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf38d3-c6ce-49a6-b4fa-ac5d733d2947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:17.375380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2ecf38d3-c6ce-49a6-b4fa-ac5d733d2947 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:17.376567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2ecf38d3-c6ce-49a6-b4fa-ac5d733d2947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:17.385714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 765/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:27:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:17.393145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6ed8c-e4bb-401a-989d-7b2ee20da054 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:17.406344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6ed8c-e4bb-401a-989d-7b2ee20da054 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:17.411322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:17.411925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:17.412762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:17.413139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:17.421629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:17.423186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8669427-8ad9-4303-903b-779b22faaee4 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:17.433396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 756/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:21.724584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f5b5-9d72-4623-bbc1-352a263c165d None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:21.726750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f5b5-9d72-4623-bbc1-352a263c165d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:21.726750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f5b5-9d72-4623-bbc1-352a263c165d None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:21.727448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:21.733581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165da12-5365-4a9f-a510-05db2ad38699 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:21.734028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165da12-5365-4a9f-a510-05db2ad38699 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:21.734203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165da12-5365-4a9f-a510-05db2ad38699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:21.754194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c165da12-5365-4a9f-a510-05db2ad38699 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:21.755395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c165da12-5365-4a9f-a510-05db2ad38699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:21.763402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:27:21.918175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7f5b5-9d72-4623-bbc1-352a263c165d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:21.932794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7f5b5-9d72-4623-bbc1-352a263c165d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:21.938101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:21.938807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:21.939086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:21.939356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:21.947984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:21.949613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b084780-2612-43a3-bb1e-01c0689e683b service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:21.959370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 767/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:27:23.062004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4a84-0712-4589-b4ce-a71155b34d8f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:23.062517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4a84-0712-4589-b4ce-a71155b34d8f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:23.062635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4a84-0712-4589-b4ce-a71155b34d8f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:23.063959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:23.070236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f6a4-35d1-402e-81d7-eb69895befc7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:23.070686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f6a4-35d1-402e-81d7-eb69895befc7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:23.070794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f6a4-35d1-402e-81d7-eb69895befc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:23.092183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4988f6a4-35d1-402e-81d7-eb69895befc7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:23.093399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4988f6a4-35d1-402e-81d7-eb69895befc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:23.102094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:23.254753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4a84-0712-4589-b4ce-a71155b34d8f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:23.269220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4a84-0712-4589-b4ce-a71155b34d8f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:23.275243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:23.275842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:23.276123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:23.276440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:23.284498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:23.286064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeba08-2d09-49f2-b683-e3a0014fa8c7 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:23.296475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 759/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:27.308337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:27.308930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:27.308930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:27.310207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:27.315497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea744e7-4fa4-4d43-a692-e5756468058c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:27.315941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea744e7-4fa4-4d43-a692-e5756468058c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:27.316049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea744e7-4fa4-4d43-a692-e5756468058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:27.336455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2ea744e7-4fa4-4d43-a692-e5756468058c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:27.337648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2ea744e7-4fa4-4d43-a692-e5756468058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:27.345435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:27.635703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:27.637417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:27.714488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1060d8-776a-4a9d-9407-1d52973f51db None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:27.718163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:27.718765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:27.719042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:27.719314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:27.728046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:27.729533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4d832-c732-4761-836e-d666affad7ab service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:27.738966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 770/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:29.871086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:29.871628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:29.871751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:29.873038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:29.878306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c732aa-67ec-432f-b827-b5a78f776161 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:29.878725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c732aa-67ec-432f-b827-b5a78f776161 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:29.878826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c732aa-67ec-432f-b827-b5a78f776161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:29.898644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-66c732aa-67ec-432f-b827-b5a78f776161 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:29.899829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-66c732aa-67ec-432f-b827-b5a78f776161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:29.907687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:30.195296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:30.196988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:30.204467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=75 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:27:30.211598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd886a-7b88-4635-99d4-4e1bcb35f205 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:30.215227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:30.215830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:30.216107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:30.216414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:30.225399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:30.226810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f694194-d53a-43ab-a5cd-413411e8f23d service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:30.236059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 762/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:27:33.821277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:33.821875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:33.821875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:33.823147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:33.828311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78bddb-0c0d-42aa-aa50-1226af16b5d9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:33.828709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78bddb-0c0d-42aa-aa50-1226af16b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:33.828861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78bddb-0c0d-42aa-aa50-1226af16b5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:33.848693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2a78bddb-0c0d-42aa-aa50-1226af16b5d9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:33.849982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2a78bddb-0c0d-42aa-aa50-1226af16b5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:33.857684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:27:34.157672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:34.159343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:34.224629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:27:34.238838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f08f-31ca-44f6-9172-7f2a7cd5a411 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:34.242948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:34.243543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:34.243816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:34.244083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:34.252769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:34.254180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48584c-2887-4b67-84bc-2e24afec8a77 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:34.263308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 773/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:36.390436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:36.390948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:36.390948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:36.392305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:27:36.397921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff48a4-611d-4fb1-b2d0-a274bb4b16e3 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:36.398358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff48a4-611d-4fb1-b2d0-a274bb4b16e3 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:36.398454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff48a4-611d-4fb1-b2d0-a274bb4b16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:36.417960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-21ff48a4-611d-4fb1-b2d0-a274bb4b16e3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:27:36.419126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-21ff48a4-611d-4fb1-b2d0-a274bb4b16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:36.426776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:27:36.733806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:27:36.735479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:27:36.751088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f00f4-c0ee-41a2-98fd-1101c1c77488 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:27:36.754947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:27:36.755785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:27:36.756096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:27:36.756417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:27:36.765475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:27:36.767053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82089fa-c4bb-4802-bfee-73ea9b1001e7 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:27:36.776867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 765/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:27:46.413244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=11 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:27:46.772792 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=14 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:02.507288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.507924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.507924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.536570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:02.537812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:02.548137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 775/1540] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:02.557207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.558039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.558039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.592445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:02.593654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:02.606582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 766/1541] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:02.612286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.626462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48366b3a-969e-41c3-a486-cd987beafddd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.635058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.635678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.635954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.636223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.636898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:02.637196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:02.638856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:02.658003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 776/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:02.667318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.683223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9a3fe-f9d8-483b-87a4-56f3ac7a251a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.688344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.688987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.689265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.689532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.690184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:02.692663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:02.749117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 767/1543] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:02 2022] POST /identity/v3/projects => generated 403 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:02.753136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.770654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479cac7-ab54-4ba0-becf-5ad1fa73dd10 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.842994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.843854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.844134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.844436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.845080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:02.847222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:02.875660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 777/1544] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:02 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:02.879895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.893720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65132c2e-e6de-4d7e-9b98-eb4a1221ef19 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.902469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.903088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.903363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.903630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.904281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:02.905904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:02.920031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 768/1545] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:02.924355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.943506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94adf-5352-4544-ae8f-5c432daaebef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.953071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.953692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.953971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.954241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:02.955003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:02.956641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ef265-7829-4dd3-9f69-559a07f66e24 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:02.970780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 778/1546] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:29:02.975207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:02.989850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30342ef5-b372-4fd9-8161-e3c214a717d4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:02.998595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:02.999271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:02.999546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:02.999813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] PATH_INFO: `/v3/projects/683fe26dd18740ebb1093f9539c75371/users/7c2c3f4186a34912862f1902fe5e3961/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.027156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683fe26dd18740ebb1093f9539c75371, user_id=7c2c3f4186a34912862f1902fe5e3961, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.029752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34dd8d-5729-4892-9ce8-2ffe4cd24675 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.049502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 769/1547] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:02 2022] PUT /identity/v3/projects/683fe26dd18740ebb1093f9539c75371/users/7c2c3f4186a34912862f1902fe5e3961/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:03.056411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb324-0f3a-4d7c-a588-1725ba1daa68 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.056752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb324-0f3a-4d7c-a588-1725ba1daa68 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.057018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb324-0f3a-4d7c-a588-1725ba1daa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.071097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce807dc5-0fc3-4dc7-ad90-3e19afde0aff None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.071452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce807dc5-0fc3-4dc7-ad90-3e19afde0aff None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.071641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce807dc5-0fc3-4dc7-ad90-3e19afde0aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.093347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6e7eb324-0f3a-4d7c-a588-1725ba1daa68 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.094647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e7eb324-0f3a-4d7c-a588-1725ba1daa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.107779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ce807dc5-0fc3-4dc7-ad90-3e19afde0aff None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.109038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ce807dc5-0fc3-4dc7-ad90-3e19afde0aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.209731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 779/1548] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.211818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae816f8-8397-4797-9415-06b12b031d82 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.212372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae816f8-8397-4797-9415-06b12b031d82 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.212857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae816f8-8397-4797-9415-06b12b031d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.243739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 770/1549] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.245467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.245840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.246171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.248487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fae816f8-8397-4797-9415-06b12b031d82 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.250055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fae816f8-8397-4797-9415-06b12b031d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.262891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 780/1550] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.264238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491159-e8b4-4188-86f2-b2494570dcc9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.264702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491159-e8b4-4188-86f2-b2494570dcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.264981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491159-e8b4-4188-86f2-b2494570dcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.273611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.275777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.284794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 771/1551] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.285221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b1491159-e8b4-4188-86f2-b2494570dcc9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.285344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.286422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b1491159-e8b4-4188-86f2-b2494570dcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.301080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 781/1552] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.302685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefb45-7464-4cb2-83e0-b00604463afb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.303149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefb45-7464-4cb2-83e0-b00604463afb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.303947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefb45-7464-4cb2-83e0-b00604463afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.327891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-79fefb45-7464-4cb2-83e0-b00604463afb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.329127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-79fefb45-7464-4cb2-83e0-b00604463afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.337435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 782/1553] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.338557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb463d5-a8fd-4e5b-97e9-ee12e5495124 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.338840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb463d5-a8fd-4e5b-97e9-ee12e5495124 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.339112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb463d5-a8fd-4e5b-97e9-ee12e5495124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.348809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523f680-2ff2-4aa1-bec1-c4e18e70eceb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.353138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.353861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.354144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.354414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.358661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1fb463d5-a8fd-4e5b-97e9-ee12e5495124 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.359846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1fb463d5-a8fd-4e5b-97e9-ee12e5495124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.364108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.365631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb0783-9e3d-425c-9446-0f2093485d45 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.368656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 783/1554] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.369824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba381c-68c8-4395-ac2c-ba20abbe4fdb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.370106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba381c-68c8-4395-ac2c-ba20abbe4fdb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.370382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba381c-68c8-4395-ac2c-ba20abbe4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.376521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 772/1555] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:03.377837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45769412-1ccf-489c-988e-723949f685a6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.378123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45769412-1ccf-489c-988e-723949f685a6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.378393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45769412-1ccf-489c-988e-723949f685a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.392925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2bba381c-68c8-4395-ac2c-ba20abbe4fdb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.394363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2bba381c-68c8-4395-ac2c-ba20abbe4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.399281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-45769412-1ccf-489c-988e-723949f685a6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.400466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-45769412-1ccf-489c-988e-723949f685a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.403760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 784/1556] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.405082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efaca-93c4-4f95-95c9-98f4288032eb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.405362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efaca-93c4-4f95-95c9-98f4288032eb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.405640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efaca-93c4-4f95-95c9-98f4288032eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.411023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 773/1557] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.412228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0493-57fd-46b4-b5ff-aaad9c65116a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.412552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0493-57fd-46b4-b5ff-aaad9c65116a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.412828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df0493-57fd-46b4-b5ff-aaad9c65116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.429365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e29efaca-93c4-4f95-95c9-98f4288032eb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.430556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e29efaca-93c4-4f95-95c9-98f4288032eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.433829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f9df0493-57fd-46b4-b5ff-aaad9c65116a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.434991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f9df0493-57fd-46b4-b5ff-aaad9c65116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.438827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 785/1558] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.439847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901cb62-88a8-4562-8bad-f6d94cd9b01c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.440126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901cb62-88a8-4562-8bad-f6d94cd9b01c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.440427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901cb62-88a8-4562-8bad-f6d94cd9b01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.446068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 774/1559] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.447386 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.447736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.448087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.460998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e901cb62-88a8-4562-8bad-f6d94cd9b01c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.462328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e901cb62-88a8-4562-8bad-f6d94cd9b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.469567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.470813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.473126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 786/1560] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.480388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.480388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.480388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.488418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 775/1561] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.489239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.495459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:03.500578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433398a9-7c66-4952-8e83-8d8d4e02b3d0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.501849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.503194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.506872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.507629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.507968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.508344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.509092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.509456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.510527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:03.510527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:29:03.510527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:29:03.511483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 787/1562] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.511998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.512212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.525175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb12d25-a6ac-4a4b-99a6-f254e36dfeff None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.532196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 776/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.532196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.532427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.532427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.532427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.532427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.532871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.533177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.534985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.535662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb15d29-c433-4b57-a249-5a6a20a2b731 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.537142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:03.542666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.543347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.543745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.544083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.544815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.545718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.547881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da155f14-ee39-4854-b451-a18ad1e09b0c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.558569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 788/1564] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.558569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.562154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 777/1565] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.563492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.563781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.564057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.571334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa413ec-dd5f-436b-9c29-b9df5bcac7df admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.576778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.577630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.577975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.578321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.579517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.579517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.581370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.587406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.588647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.604305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 789/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.604305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.604652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 778/1567] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:03.604652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.613820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9ffbc-ce22-459b-8b9a-2b95b97d3b1b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.618943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.619553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.619778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04607a-9f92-4327-8104-a0fc183b620e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.619892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.620080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.620871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.621049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.622751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.627971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.629607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.630365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.631138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.632922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.633419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.642921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.643210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 779/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.643210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.656380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a865f59-660a-43b4-920d-dc580f512255 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.657608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 790/1569] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.659225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.665413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.666039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.666315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.666587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.667275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.669196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.680958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52408cb7-1eef-44df-b97d-54dc23482fdf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.686688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.687501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.687847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.688182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.689039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.690373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:03.690373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:29:03.690373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:29:03.691882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.708099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 780/1570] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:03.712349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.725600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 791/1571] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 407 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:03.727549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.786711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d2a68-50f9-412d-a393-2f3c39971608 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.813029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a0f3-166b-4047-a8db-14c6697ba31e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.818577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.820373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.820971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.822045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.823254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:03.823781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.825888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.841313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 792/1572] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:03.842508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.856554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7607ed-ec4f-4184-9f2b-1e579e1d2d77 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.864413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.865000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.865361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.865593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.865884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.866048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.866610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.866917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.867251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.868028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.869370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.870498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.905305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 793/1573] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:03.905305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.907394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 781/1574] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 393 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:03.908390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.941990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:03.980820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb7015-9792-4cf1-85a6-cfe8925d9354 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.981334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344402-c491-4bdc-9245-fc6f7fe3d2eb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:03.986270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.987048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.987419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.987748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:03.988029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.988683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.988927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.989100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.989510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.990481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:03.991258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:03.993597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.026254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 782/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.027081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.031124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 794/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.032170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.038430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.107994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5688b-1c68-4268-b79f-52d48a296575 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.143749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdc2d1-d53f-42bd-ad49-fef5c3b6a0d7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.149235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.149935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.150591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.150591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.151253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.153572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.180121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 795/1577] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 304 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.181528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.188385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.189012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.189294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.189567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.190208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.191277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:04.191277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:29:04.191277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:29:04.192549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.196582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc3b0-a1bd-4f07-85e7-0b5744fa3848 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.201695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.202307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.203053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.203336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.204553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.207341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.224046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 783/1578] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.224046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.233810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417484c-6f0c-44d4-a12b-7cd22e228925 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.239266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.240026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.240350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.240628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.241260 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.244237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.244899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 796/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.248944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.269184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 784/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.270428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.284775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cf1f4-ef4e-4c2f-9216-d52dfb074d1e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.323066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad6c44-0c36-49ac-ad8f-60876ff4ffa1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.324549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.328472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.329238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.329522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.329873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.330602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.332593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.353242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.353860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.354073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 797/1581] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.354444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.354775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.354932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.355362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.357342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655efbc-d386-4e47-a40c-2aab6733a6de admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.365026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8beead-049f-4372-b871-bde8bb2536b8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.371465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.372181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.372500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.372764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.373633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.375531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.379590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 785/1582] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.384370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.384669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.384945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.398492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 798/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.399430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.408850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eb58b-59a9-4c73-b5f5-becfd935ca2c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.413488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.414275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.414553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.414823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.415447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.416742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] MFA Rules not processed for user `7c2c3f4186a34912862f1902fe5e3961`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:04.417063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.417912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:04.427359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 799/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.428131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.437773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113955ad-d1a0-403b-bda7-51a2efbccb7a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.442582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.443534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.443811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.444078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.445439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.446953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.456507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 800/1585] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.456507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.467256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c26e-702c-47fe-a6ff-166736f6f10e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.471759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.472361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.472638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.472905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.473507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.475363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.484356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 786/1586] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:04.485059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.497517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faebe5f-da54-45f7-a989-dc5edef844ae None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.503848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 801/1587] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.503848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.504100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.504100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.504100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.504100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.504476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.505722 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.506493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.513766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f734a78-b69b-48a1-8cf1-2b531ce45958 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.516854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 787/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.517729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.518679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.519277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.519796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.519974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.520633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.522232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.527383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a19d0-c5ae-4263-9b98-26c3e1ce6b0d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.531888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.532499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.532781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.532978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 802/1589] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.533091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.533726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.533911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.535289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.544801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df63792-3141-4ec7-8b91-945309fa2d4b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.546084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 788/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.546870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.549701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.550310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.550582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.550850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.551468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.553067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.557348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b8ca-145f-4a19-ac21-044f55877437 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.562098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.562680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.563217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.563329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.563887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.567000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 803/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.567153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.567249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.575384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de9f9-278f-4ac6-add0-46c7597c6d6d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.576776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 789/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.577507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.580497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.581089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.581373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.581648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.582277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.583862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.587432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941512-4f3d-4437-97b8-70e2c8710666 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.592031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.592639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.592916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.593193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.593815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.597238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 804/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.597238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.597476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.606456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f56f0c-15d4-4188-a5ea-bf681fdaf796 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.611328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.611936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.612310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.612522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.613163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.614891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.627893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 790/1594] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.627893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.628205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 805/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.628205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.637446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d0509-e9ee-4aad-a668-756d9a9cfa69 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.688982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.693016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5efd0d-b565-44a8-919d-c206a18887b9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.695073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.696070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.696413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.696664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.697335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.698915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.699049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.699518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.699798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.700107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.700778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.702392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.708834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 806/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.709631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.715600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 791/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.716351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.719881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923aa6-db24-4f73-b97c-621138e76953 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.724534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.725120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.725396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.725657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e388-f832-40ce-adf0-3c673ce5087c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.726301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.727066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.728716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.730827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.731404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.731683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.731958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.732617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.734174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.738717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 807/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.739592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.746760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 792/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.747743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.749789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68818e01-f08e-4520-bc82-918db9c56355 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.754287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.754923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.755205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.755477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] PATH_INFO: `/v3/projects/cce8591f14484fe2ba7ae74c07222b01/users/ab90ebcd18bf44bd866edef42ef7fbfe/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.757041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aee08-c2f9-4184-b011-a760e69a8ce8 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.761700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.762328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.762605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.762874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] PATH_INFO: `/v3/projects/0a676eb5cbd74956ba12b1d5595e3f18/users/4056ea3d1a1d48f2a34e14df77914363/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.770619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce8591f14484fe2ba7ae74c07222b01, user_id=ab90ebcd18bf44bd866edef42ef7fbfe, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.771647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:04.771647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:29:04.771647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:29:04.773804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.777148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a676eb5cbd74956ba12b1d5595e3f18, user_id=4056ea3d1a1d48f2a34e14df77914363, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.780655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.787232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 808/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:04 2022] PUT /identity/v3/projects/cce8591f14484fe2ba7ae74c07222b01/users/ab90ebcd18bf44bd866edef42ef7fbfe/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:04.788071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.805076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 793/1601] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:04 2022] PUT /identity/v3/projects/0a676eb5cbd74956ba12b1d5595e3f18/users/4056ea3d1a1d48f2a34e14df77914363/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:04.806063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.852686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ab745-dfed-433a-b1ff-f82590f9ed68 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.865652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2ca91-fd5f-431b-91e1-0b18a8fabf31 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.870413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.871058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.871332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.871594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.872217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.873944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.876262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.884173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 794/1602] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:04.885117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.895831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f71519-84b6-4880-8c08-b612cf8ed996 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.901118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.901745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.902023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.902288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] PATH_INFO: `/v3/projects/01c8f7766cb646b7b646a53c9de8e91a/users/683c4d41fa4a452ba8210e62dd43f7f4/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.912930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.914197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.914469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.914733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.915352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.917368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.917644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c8f7766cb646b7b646a53c9de8e91a, user_id=683c4d41fa4a452ba8210e62dd43f7f4, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.920334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.934388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 795/1603] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:04 2022] PUT /identity/v3/projects/01c8f7766cb646b7b646a53c9de8e91a/users/683c4d41fa4a452ba8210e62dd43f7f4/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:04.935242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.940484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 809/1604] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 308 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.941463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.952716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca790cf7-8731-48aa-a7b3-d5a4a9b998a6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.001886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e308-f87f-4056-ad33-7f1f28939302 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.006531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.007311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.007596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.007874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] PATH_INFO: `/v3/projects/e1868f47f7ab43d5be20bb69022d51fb/users/560af8d93f924c298f7687f249585ce4/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.008570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.009227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.009613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.009961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] PATH_INFO: `/v3/projects/fd13dd8ec810435491e3f3cba7cca4d1/users/48e588ff6d5a4a37b902abadfb6d04fc/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.022394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1868f47f7ab43d5be20bb69022d51fb, user_id=560af8d93f924c298f7687f249585ce4, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.024790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd13dd8ec810435491e3f3cba7cca4d1, user_id=48e588ff6d5a4a37b902abadfb6d04fc, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.025773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.027458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e503-2dff-40e6-8759-11dff7694513 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.040027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 796/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:04 2022] PUT /identity/v3/projects/e1868f47f7ab43d5be20bb69022d51fb/users/560af8d93f924c298f7687f249585ce4/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:05.040654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 810/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:04 2022] PUT /identity/v3/projects/fd13dd8ec810435491e3f3cba7cca4d1/users/48e588ff6d5a4a37b902abadfb6d04fc/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:05.041717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf138c-bfe7-4a0b-b552-572c667ca9b2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.042031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf138c-bfe7-4a0b-b552-572c667ca9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.042300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf138c-bfe7-4a0b-b552-572c667ca9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.044153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.064161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.071489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-66cf138c-bfe7-4a0b-b552-572c667ca9b2 None None] MFA Rules not processed for user `ab90ebcd18bf44bd866edef42ef7fbfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.072732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-66cf138c-bfe7-4a0b-b552-572c667ca9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.115929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9a38-0c29-4f8f-bb95-a0de8162111a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.141215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 811/1607] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.146019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df1bc8-3069-4fa9-8773-22e6bc80b95b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.146308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df1bc8-3069-4fa9-8773-22e6bc80b95b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.146584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df1bc8-3069-4fa9-8773-22e6bc80b95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.165280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-03df1bc8-3069-4fa9-8773-22e6bc80b95b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.166626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-03df1bc8-3069-4fa9-8773-22e6bc80b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.171964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.172710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.173021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.173316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] PATH_INFO: `/v3/projects/e666edf14c434665bec2a5f89efffc8c/users/3dca1fc7b21148639d328bc371158494/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.188100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e666edf14c434665bec2a5f89efffc8c, user_id=3dca1fc7b21148639d328bc371158494, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.190825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.190966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.205084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 797/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:05 2022] PUT /identity/v3/projects/e666edf14c434665bec2a5f89efffc8c/users/3dca1fc7b21148639d328bc371158494/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:05.205900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.238310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 812/1609] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.239566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.239851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.240123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.260208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.261423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.269212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 813/1610] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.269441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f68a-0a20-4ff9-bcbe-a1184a5bccf4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.273149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.274263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.274917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.275192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.275466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] PATH_INFO: `/v3/projects/e5e1c15ae90e4b82b4e216b3a0549e2e/users/a8f59b47841a4b62b8e368562c7bd210/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.282984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799358b-72f1-4182-be51-77e43da5e3a4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.287734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.288988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.289268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.289560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.290189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.291112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e1c15ae90e4b82b4e216b3a0549e2e, user_id=a8f59b47841a4b62b8e368562c7bd210, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.291793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9543319-713a-4691-9a4a-3456ce45a6ee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.293770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94498fb-f83c-4959-a4ae-4f6196b64e4f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.302361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 814/1611] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:05.303631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.304488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.304929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.311319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 798/1612] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:05 2022] PUT /identity/v3/projects/e5e1c15ae90e4b82b4e216b3a0549e2e/users/a8f59b47841a4b62b8e368562c7bd210/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:05.314921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260ae4-073a-42bf-b351-c1e17fdeff03 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.314921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260ae4-073a-42bf-b351-c1e17fdeff03 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.314921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260ae4-073a-42bf-b351-c1e17fdeff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.323982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.325204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.340478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-17260ae4-073a-42bf-b351-c1e17fdeff03 None None] MFA Rules not processed for user `560af8d93f924c298f7687f249585ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.341663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-17260ae4-073a-42bf-b351-c1e17fdeff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.354873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.386050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 815/1613] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.386946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.397278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aae7e4-5922-48ae-956a-198ab1302384 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.402132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.402725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.403000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.403287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.403884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.405805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.409265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 799/1614] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.410527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.410913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.411250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.434824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 816/1615] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:05.435742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.441102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] MFA Rules not processed for user `3dca1fc7b21148639d328bc371158494`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.442343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.490908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5983-1ee8-46a7-9bff-94eed3df010c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.494779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.495374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.495652 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.495917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.508691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 800/1616] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.509478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.518936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ee576-9c82-41e8-827e-25273a967dd8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.522671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.523905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.524243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.524545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.549842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.568182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.570201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.580498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 817/1617] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:05.581258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.590299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.591362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f83b7d-4567-4ed7-afd4-3687c80ecfe4 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.591747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd27da-1bac-48fc-a010-ca1ab8fc9135 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.595975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.596642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.596923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.597194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] PATH_INFO: `/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76/users/b89b730b62f043f9a464b7f069c0ff4a/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.603532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 801/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:05.605419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.605602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.605887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.614978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fc76e6ae294493b3a971b3b4a92a76, user_id=b89b730b62f043f9a464b7f069c0ff4a, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.618362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.628401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.629606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.633240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 818/1619] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:05 2022] PUT /identity/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76/users/b89b730b62f043f9a464b7f069c0ff4a/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:05.634493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.677875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.704778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c45a-5c67-4d1c-ace4-88036cd428e5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.708709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.709321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.709621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.709871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.718219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 802/1620] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:05.718219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.728923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ba79d-c095-47c4-8ec4-b7e7f1d87161 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.730607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.732986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.734369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.735082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.735445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.736062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.736876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.739580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad985d1d-0f32-45c0-9ea4-02d044b2aed2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.745199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 819/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:05.746309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.757820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a52fe-f14d-46e7-83de-1c2368f532eb service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.765139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.765694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.765971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.766250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.767274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.770360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.791755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 803/1622] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:05.791755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2830-674a-4946-8bed-8666b73b58e8 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.791755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2830-674a-4946-8bed-8666b73b58e8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.791755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2830-674a-4946-8bed-8666b73b58e8 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.792734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 804/1623] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:05.799523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2830-674a-4946-8bed-8666b73b58e8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.805959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 820/1624] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/users => generated 311 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:05.807496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.825916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4b2be-e919-419f-9041-380fe6cf6145 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.897115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2830-674a-4946-8bed-8666b73b58e8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.903340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.904060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.904379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.904512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.904647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.905249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.905535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.905651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.905803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.909567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.922117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.924441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 805/1625] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:05.925340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.928006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ec265-3282-4e86-83fb-90ccff88f644 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.938147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2980-fa1f-43b1-93c1-c56888eafc87 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.943752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.944061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 821/1626] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:05.944399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.944767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.945045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.945655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.945776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.945944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.946221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.948207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.978573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 806/1627] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:05.979785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.015515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa288d-1199-4f8c-95eb-6c1ba0fc3065 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.023564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:06.025579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:06.030810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.031571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.031901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.032479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.033245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.036073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.067002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 807/1628] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:06.067002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.068434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.091991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755408e-c3da-427e-acbc-b4aae37415ed admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.107769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.108640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.108916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.109223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] PATH_INFO: `/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76/users/b89b730b62f043f9a464b7f069c0ff4a/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.115192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fc76e6ae294493b3a971b3b4a92a76, user_id=b89b730b62f043f9a464b7f069c0ff4a, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.124447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.131179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 822/1629] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:06.132225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.143330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e4b1-c844-4d44-81b1-e1d36c0c5e19 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.149458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.150105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.150420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.150694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.151616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.153374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.161387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 808/1630] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:06 2022] PUT /identity/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76/users/b89b730b62f043f9a464b7f069c0ff4a/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:06.161387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.168048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 823/1631] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:06.168048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.186883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840dad4-fa5c-4a7e-9f1b-f1d9c8c8801b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.248291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc612b9-39c5-4263-bad6-1a67285eaa7b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.254894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.255568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.255843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.256111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] PATH_INFO: `/v3/projects/0a2db628604842f1bcec89c78aca3b43/users/5ba6f57d30d4459ea10f852965534866/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.281401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2db628604842f1bcec89c78aca3b43, user_id=5ba6f57d30d4459ea10f852965534866, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.284455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.284608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.285239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.286225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.286566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.289145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.309072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 809/1632] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:06 2022] PUT /identity/v3/projects/0a2db628604842f1bcec89c78aca3b43/users/5ba6f57d30d4459ea10f852965534866/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:06.309905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.381274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.385909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.413223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 824/1633] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:06.417045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.440238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc4488-2424-4abc-aee1-047c4f9f46a4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.442344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123c4a9-6f99-4956-bbe9-5b843c1b18ab service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.454034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.455893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.456183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.456490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] PATH_INFO: `/v3/projects/3e4e6b488b75450da9c844cd0499fd34/users/429805a9fb4e4c9c950552f5cf07df3e/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.457142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.459234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.460694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.462065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.463694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.469302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.506415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 825/1634] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:06.509502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.522353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4e6b488b75450da9c844cd0499fd34, user_id=429805a9fb4e4c9c950552f5cf07df3e, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.532272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.537789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978eb07-0f94-4432-bf21-a736196c62b7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.554681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.557283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.558970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.560071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] PATH_INFO: `/v3/projects/0a2db628604842f1bcec89c78aca3b43/users/5ba6f57d30d4459ea10f852965534866/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.565020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2db628604842f1bcec89c78aca3b43, user_id=5ba6f57d30d4459ea10f852965534866, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.569232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 810/1635] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:06 2022] PUT /identity/v3/projects/3e4e6b488b75450da9c844cd0499fd34/users/429805a9fb4e4c9c950552f5cf07df3e/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:06.571859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1070fb6-252e-42ea-bc83-37f21b16f57d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.571998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.604317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 826/1636] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:06 2022] PUT /identity/v3/projects/0a2db628604842f1bcec89c78aca3b43/users/5ba6f57d30d4459ea10f852965534866/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:06.618986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.619401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.619770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.689324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] MFA Rules not processed for user `5ba6f57d30d4459ea10f852965534866`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:06.691947 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:06.718406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.744825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c885b-529c-4757-864a-2223f07bc4ac admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.869564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 827/1637] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:06 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:06.878620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.878993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.879626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.879910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.880180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.881467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.883719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.910983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 811/1638] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:06.916048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.947794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2502-82dc-405a-bc2e-60c4c18be06e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.954523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.955254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.955539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.955830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] PATH_INFO: `/v3/projects/3e4e6b488b75450da9c844cd0499fd34/users/429805a9fb4e4c9c950552f5cf07df3e/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.958710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4e6b488b75450da9c844cd0499fd34, user_id=429805a9fb4e4c9c950552f5cf07df3e, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.962202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793f9a6-9f15-44c4-a100-30f93d2c6221 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.984290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0abc-6b4a-412a-a812-4cb13b2c3d44 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.984719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 812/1639] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:06 2022] PUT /identity/v3/projects/3e4e6b488b75450da9c844cd0499fd34/users/429805a9fb4e4c9c950552f5cf07df3e/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:06.986015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a39a-02e9-49be-8634-dfc2b5e9400b None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.986379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a39a-02e9-49be-8634-dfc2b5e9400b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.986669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a39a-02e9-49be-8634-dfc2b5e9400b None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.994906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 813/1640] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 05:29:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:06.996206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86019e-8c58-45ad-aace-443a71292052 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.996575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86019e-8c58-45ad-aace-443a71292052 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.996854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86019e-8c58-45ad-aace-443a71292052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.030577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.050951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-de86019e-8c58-45ad-aace-443a71292052 None None] MFA Rules not processed for user `429805a9fb4e4c9c950552f5cf07df3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:07.052311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-de86019e-8c58-45ad-aace-443a71292052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:07.099246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.100053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.100192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.100544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.180450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 814/1641] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:07.185995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.186659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.187114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.197458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.198926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.226653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 828/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:07.229217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.231081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:07.232796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:07.240317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71507-40ef-41c1-8593-b76c46f13d7d service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.244776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.245994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.246285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.246650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.257913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.260065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.283443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 829/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:07.333668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 815/1644] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:29:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:07.386843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.392786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.402635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6913a-0dfc-4ce7-82ae-af9315df4be7 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.408433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.410036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.410528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.411114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.426486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.428582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.460733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 830/1645] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:07.633562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.645437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bf491-6ca2-41ef-876e-e7b39af75eed None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.658285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.658932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.660006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.660006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.796803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.802905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86ff0-2f34-410a-b596-fcb7bfb99b8f service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.833643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 816/1646] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:08.246188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.258822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4a28-a15f-40bf-9edb-06d45b587f89 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.269514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.271607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.271607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.271607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.428362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.430847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6248f6e-0f0d-451b-8e18-101276b3eca5 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.455093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 831/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:08.988920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b126-5660-456f-8f47-223395ab6876 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.989866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b126-5660-456f-8f47-223395ab6876 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.990827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b126-5660-456f-8f47-223395ab6876 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.993115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 817/1648] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:29:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:09.008192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46659f66-f463-4a97-9ed5-b40b961edee6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.008502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46659f66-f463-4a97-9ed5-b40b961edee6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.008979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46659f66-f463-4a97-9ed5-b40b961edee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.064173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-46659f66-f463-4a97-9ed5-b40b961edee6 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:09.065440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-46659f66-f463-4a97-9ed5-b40b961edee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:09.164010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 832/1649] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:09.177866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9192-9e82-45d7-ae66-54544c75552f None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.177866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9192-9e82-45d7-ae66-54544c75552f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.177866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9192-9e82-45d7-ae66-54544c75552f None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.177866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 818/1650] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 05:29:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:09.179535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c41302-f409-4b82-836d-6e744bb5713c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.179535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c41302-f409-4b82-836d-6e744bb5713c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.179535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c41302-f409-4b82-836d-6e744bb5713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.213326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f0c41302-f409-4b82-836d-6e744bb5713c None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:09.214425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f0c41302-f409-4b82-836d-6e744bb5713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:09.246061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 833/1651] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:09.399116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda9192-9e82-45d7-ae66-54544c75552f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.408551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda9192-9e82-45d7-ae66-54544c75552f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.417132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.419362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.419779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.420488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.443043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.444523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3610c71-8aab-4261-9571-f59507b09c73 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.466922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 819/1652] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:09.497666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40958d3-00aa-45da-852a-104af3eaec5c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.498138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40958d3-00aa-45da-852a-104af3eaec5c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.498488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40958d3-00aa-45da-852a-104af3eaec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.576970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a40958d3-00aa-45da-852a-104af3eaec5c None None] MFA Rules not processed for user `b89b730b62f043f9a464b7f069c0ff4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:09.580100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a40958d3-00aa-45da-852a-104af3eaec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:09.821771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 834/1653] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:12.783723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429ead1-e8c0-48ad-9c65-ae6e451b756e None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.785802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429ead1-e8c0-48ad-9c65-ae6e451b756e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.786257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429ead1-e8c0-48ad-9c65-ae6e451b756e None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.788530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 820/1654] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:29:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:12.848189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.850768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.851405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.895586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:12.904060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:13.060208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:13.087664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 835/1655] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:13.174789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429ead1-e8c0-48ad-9c65-ae6e451b756e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:13.216638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429ead1-e8c0-48ad-9c65-ae6e451b756e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:13.234316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:13.234944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.235220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.235487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.264982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:13.266516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7b943-5d8b-483b-85be-715f2a47d83a service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:13.280774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:13.300513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 821/1656] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:16.381155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.520411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e134c4f-0845-4c0b-9eca-ba76ec4aa3e7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.539488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.540946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.541375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.542260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] PATH_INFO: `/v3/users/7c2c3f4186a34912862f1902fe5e3961` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.543848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2c3f4186a34912862f1902fe5e3961)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.547359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: WARNING py.warnings [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:16.547359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: warnings.warn(msg) Sep 26 05:29:16.547359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: Sep 26 05:29:16.550531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.717206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2c3f4186a34912862f1902fe5e3961'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:16.720382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2c3f4186a34912862f1902fe5e3961'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:16.742544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2c3f4186a34912862f1902fe5e3961'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:16.743452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2c3f4186a34912862f1902fe5e3961'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:16.772242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-52cdf419-9f62-4582-a1e9-605c4cf54421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2c3f4186a34912862f1902fe5e3961'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:16.828757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 836/1657] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:16 2022] DELETE /identity/v3/users/7c2c3f4186a34912862f1902fe5e3961 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:17.168500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.169374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.169820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.207186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] MFA Rules not processed for user `4056ea3d1a1d48f2a34e14df77914363`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:17.208914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:17.383933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 822/1658] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:17.386468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.386955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.387155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.452434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] MFA Rules not processed for user `4056ea3d1a1d48f2a34e14df77914363`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:17.454238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:17.491166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 837/1659] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:17.504922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:17.507606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:17.697655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd62e9-a5e8-40df-ad42-c3d1149ccac3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:17.708577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:17.709205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.709610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.709781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.721329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:17.723016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:17.745965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:17.747429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:17.766872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 823/1660] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:17.907334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fcf7b-d379-4926-96a7-58a186f06bc0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:17.912990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:17.913572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.913844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.914111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] PATH_INFO: `/v3/projects/683fe26dd18740ebb1093f9539c75371` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.918803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683fe26dd18740ebb1093f9539c75371)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:17.921019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:17.941738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:18.027896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683fe26dd18740ebb1093f9539c75371', 'request_id': 'req-dd442b1f-b359-41aa-bd57-8756b4b340e7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.056512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683fe26dd18740ebb1093f9539c75371', 'request_id': 'req-dd442b1f-b359-41aa-bd57-8756b4b340e7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.068920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683fe26dd18740ebb1093f9539c75371', 'request_id': 'req-dd442b1f-b359-41aa-bd57-8756b4b340e7'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.120984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] The token cache is being invalidate because project 683fe26dd18740ebb1093f9539c75371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:18.121545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.123318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dd442b1f-b359-41aa-bd57-8756b4b340e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.126517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 838/1661] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:17 2022] DELETE /identity/v3/projects/683fe26dd18740ebb1093f9539c75371 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:18.130573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.142200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36a10e-1b2e-4779-af2a-07791e50f8fa service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.277225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.278017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.278335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.278426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] PATH_INFO: `/v3/users/b89b730b62f043f9a464b7f069c0ff4a` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.280044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89b730b62f043f9a464b7f069c0ff4a)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:18.282134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:18.324960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:18.395211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89b730b62f043f9a464b7f069c0ff4a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.407646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89b730b62f043f9a464b7f069c0ff4a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.420315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89b730b62f043f9a464b7f069c0ff4a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.438527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89b730b62f043f9a464b7f069c0ff4a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.438527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89b730b62f043f9a464b7f069c0ff4a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:18.449137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed38b0-75cb-4da3-8dc0-b3f2778b29d4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.449581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed38b0-75cb-4da3-8dc0-b3f2778b29d4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.449664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed38b0-75cb-4da3-8dc0-b3f2778b29d4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.451486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 839/1662] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 05:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:18.460865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.461142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.461435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.474895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 824/1663] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:18 2022] DELETE /identity/v3/users/b89b730b62f043f9a464b7f069c0ff4a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:18.560351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:18.561694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:18.679697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 840/1664] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:29:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 26 05:29:18.697288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.699084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:18.721692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56642e26-1d16-4a2a-a041-7990cc6ae31f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.729850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.730816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.730980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.731509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.741944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:18.745243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457644d6-044e-4713-acfb-4fb6cb27794f service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:18.762464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.764332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:18.768044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 825/1665] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:18.863881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2610a-8840-465c-8f3c-10c299a4a807 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.872791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.890222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.890969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.891420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] PATH_INFO: `/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.895183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fc76e6ae294493b3a971b3b4a92a76)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:18.899331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.017239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.031294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fc76e6ae294493b3a971b3b4a92a76', 'request_id': 'req-73855e94-83c6-4924-92b5-3f9ed4db0d06'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:19.041708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fc76e6ae294493b3a971b3b4a92a76', 'request_id': 'req-73855e94-83c6-4924-92b5-3f9ed4db0d06'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:19.048926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fc76e6ae294493b3a971b3b4a92a76', 'request_id': 'req-73855e94-83c6-4924-92b5-3f9ed4db0d06'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:19.071494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] The token cache is being invalidate because project b9fc76e6ae294493b3a971b3b4a92a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:19.072108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:19.073328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-73855e94-83c6-4924-92b5-3f9ed4db0d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:19.080354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 841/1666] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:18 2022] DELETE /identity/v3/projects/b9fc76e6ae294493b3a971b3b4a92a76 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:19.085846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.086197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.086512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.126166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:19.127408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:19.249329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 826/1667] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:19.260081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.261100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.262453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.293863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:19.295304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:19.304536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 842/1668] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:19.308493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.321869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b813260-973e-4e09-93a9-883ba18d9d75 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.326847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.328140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.328653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.329099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.329937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:19.330403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.332383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.348577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 827/1669] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:19.353028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.372682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa01f-26ed-49bc-95fa-79a5297dfe76 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.380365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.382671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.383041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.383381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.384092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.386475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.454733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 843/1670] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:19.459759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.471586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901c7d0-058a-41eb-883b-e19a30e464d3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.569571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.570194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.570702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.570702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.571374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.573454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.577849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.579669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:19.617789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.641132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 828/1671] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:29:19.645187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.665502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13cc0-d17c-465b-90b3-41bffc3d50a0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.675233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.675890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.676176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.676478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.677367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.679638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.698342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 829/1672] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:19.704490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.732186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee657-c11e-48f4-93cf-18c9627848fb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.750064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582fa3-3fae-4023-b4f5-dd0372816a12 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.754742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.755945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.757357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.757792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.759224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.759837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.760118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.760417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.761058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.761839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:19.762645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14c991-c121-4a9d-8bec-bc0f59708d7d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.777067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 830/1673] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:19.777067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fa104-9404-43cf-9ba3-400318b37ba1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.777067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fa104-9404-43cf-9ba3-400318b37ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.777067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fa104-9404-43cf-9ba3-400318b37ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.779519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.782223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.795636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 844/1674] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:19.797258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.811283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-016fa104-9404-43cf-9ba3-400318b37ba1 None None] MFA Rules not processed for user `683c4d41fa4a452ba8210e62dd43f7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:19.812521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-016fa104-9404-43cf-9ba3-400318b37ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:19.814826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f99bf9-29eb-499e-a8cd-7871d2e5dddd service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.820338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.821226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.831712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.832342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] PATH_INFO: `/v3/projects/cf1abe56cb75460798bc84ff453ed09d/users/9a8467a69b7c45348f7cf66e10f24c89/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.855286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1abe56cb75460798bc84ff453ed09d, user_id=9a8467a69b7c45348f7cf66e10f24c89, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.859114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94400b8b-29f8-41e0-b0c0-9d3d1623419f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.904265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 845/1675] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:19 2022] PUT /identity/v3/projects/cf1abe56cb75460798bc84ff453ed09d/users/9a8467a69b7c45348f7cf66e10f24c89/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:19.916597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.917294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.917707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.945743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 831/1676] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:19.955575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.956485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.957105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.988017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] MFA Rules not processed for user `9a8467a69b7c45348f7cf66e10f24c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:19.991924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:20.002808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] MFA Rules not processed for user `683c4d41fa4a452ba8210e62dd43f7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:20.005077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:20.207599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:20.268963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 832/1677] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:20.274653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 846/1678] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:20.278712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.284396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.288767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:20.487162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd860da-f0e6-40ea-b764-a070c7a4f6a0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.496074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.497104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.497629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.498139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.499230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:20.502335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:20.528963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:20.559667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 833/1679] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:20 2022] POST /identity/v3/projects => generated 371 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:20.563335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.567588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e39e4-89d2-4c06-b63a-eedf03c40507 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.821129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b305da-8105-40a7-9f66-19c76c7a6284 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.838609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.842216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.842524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.842806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.843282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.843944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:20.846331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.847473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.847746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.861799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:20.977868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 834/1680] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:20 2022] POST /identity/v3/users => generated 292 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:20.981225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.020141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3d8f9-455c-43af-bdd0-20d79aa3ab20 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.048528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.050553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.052150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.054209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.058125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.064884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.126580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 835/1681] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:21.126662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.126732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.126828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.144330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 847/1682] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:21.180605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c362b-adab-498d-85d2-a897562cde1f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.203116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.203816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.204170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.204448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] PATH_INFO: `/v3/projects/e50ecbdcbc17481894af5685223413ad/users/f924e3d498e0455c934e95deaa8648b0/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.251094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50ecbdcbc17481894af5685223413ad, user_id=f924e3d498e0455c934e95deaa8648b0, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.261564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.317454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 836/1683] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:21 2022] PUT /identity/v3/projects/e50ecbdcbc17481894af5685223413ad/users/f924e3d498e0455c934e95deaa8648b0/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:21.319741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.339210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88314be-283d-490c-8a8e-08ba581086a4 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.446381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:21.510022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.511858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:21.572663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.576718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.577595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.578927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.579979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.584861 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.642373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 848/1684] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:21.644440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.689817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e47fa-93f9-4c99-972c-4f335be4d917 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.701040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.702015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.703273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.704445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] PATH_INFO: `/v3/projects/e50ecbdcbc17481894af5685223413ad/users/f924e3d498e0455c934e95deaa8648b0/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.710198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50ecbdcbc17481894af5685223413ad, user_id=f924e3d498e0455c934e95deaa8648b0, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.719369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570033e-f0eb-46cc-a31b-67ce87d70a21 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.765526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ffd49-9b9d-4b7b-a2f0-1770215a3db6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.766008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 849/1685] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:21 2022] PUT /identity/v3/projects/e50ecbdcbc17481894af5685223413ad/users/f924e3d498e0455c934e95deaa8648b0/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:21.767669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bf0fc-932a-47bf-b375-7c9f59fcad30 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.768056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bf0fc-932a-47bf-b375-7c9f59fcad30 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.769120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bf0fc-932a-47bf-b375-7c9f59fcad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.876713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-102bf0fc-932a-47bf-b375-7c9f59fcad30 None None] MFA Rules not processed for user `f924e3d498e0455c934e95deaa8648b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:21.884142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-102bf0fc-932a-47bf-b375-7c9f59fcad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:21.987749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.031707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.032395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.032668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.032936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] PATH_INFO: `/v3/users/3dca1fc7b21148639d328bc371158494` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.036846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dca1fc7b21148639d328bc371158494)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.051229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.227587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dca1fc7b21148639d328bc371158494'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:22.262537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dca1fc7b21148639d328bc371158494'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:22.263510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dca1fc7b21148639d328bc371158494'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:22.290483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dca1fc7b21148639d328bc371158494'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:22.291125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 850/1686] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:22.298593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287711bc-b8b7-40cf-ab6c-e60cab7fffae None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.299127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4ae0e4aa-1dd9-421e-b102-8e5e8aa8e24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dca1fc7b21148639d328bc371158494'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:22.300550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287711bc-b8b7-40cf-ab6c-e60cab7fffae None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.301359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287711bc-b8b7-40cf-ab6c-e60cab7fffae None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.307307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 851/1687] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 05:29:22 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:22.320729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.321442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.322346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.385184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 837/1688] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:21 2022] DELETE /identity/v3/users/3dca1fc7b21148639d328bc371158494 => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:22.387514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.389932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.390683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.404074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:22.414643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.466240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:22.474214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.641394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 852/1689] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:22.654664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.656630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.738310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 838/1690] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:22.741615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.747540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.792760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f217e-afb2-4863-a83b-6b88ba88ad68 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.810648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.818524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.819378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.819696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.820045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.847043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab688e4-6a01-4ef7-a9ed-6cf3048fd3bc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.856475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.861150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.861869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.861973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.862123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.862601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.867307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.895561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 839/1691] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:22.901161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.903638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:22.951222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923c0c-d327-4da8-b75c-47d43e90ac10 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.975739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.976646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.976925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.977194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] PATH_INFO: `/v3/users/ab90ebcd18bf44bd866edef42ef7fbfe` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.979454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab90ebcd18bf44bd866edef42ef7fbfe)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.991017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.201438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.203578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab90ebcd18bf44bd866edef42ef7fbfe'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:23.224126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.246429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab90ebcd18bf44bd866edef42ef7fbfe'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:23.248094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab90ebcd18bf44bd866edef42ef7fbfe'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:23.252179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:23.275088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab90ebcd18bf44bd866edef42ef7fbfe'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:23.275517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 853/1692] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:23.276473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.277602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-499da991-31a1-47f5-a596-22b61ba56dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab90ebcd18bf44bd866edef42ef7fbfe'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:23.299570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a455-30dd-401a-9a51-90a5e3205495 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.342945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:23.345875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.346786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.350514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.354203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.369238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 840/1693] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:22 2022] DELETE /identity/v3/users/ab90ebcd18bf44bd866edef42ef7fbfe => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:23.382573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.388094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.389878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.422869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.425674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc82e-6111-4bf7-a060-c8a499b1b561 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.462688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 854/1694] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:23.464437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.464604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.464845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.492731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] MFA Rules not processed for user `a8f59b47841a4b62b8e368562c7bd210`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:23.502210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.534501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:23.536523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.630370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 841/1695] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:23.632781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.635369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.733682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 855/1696] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:23.735301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.738528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.821090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ac810-08bc-4844-bb2a-65fd255abeef None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.841545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:23.845131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.845764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.847347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.867358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.937269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078445d2-4905-4b54-bf52-35469c952a35 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.967510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:23.970507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.971367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.973045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.994934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.997056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.038262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 856/1697] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:24.043489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.048138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:24.093356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:24.173099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.178799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184afa41-3a1a-4081-bfd9-b4483fd5046e service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.243799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 842/1698] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:24.250929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.252377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.252661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.339770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a5b06-de4a-4261-8c87-904c44821a35 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.364307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.367624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.369526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.369962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.397765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] MFA Rules not processed for user `48e588ff6d5a4a37b902abadfb6d04fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:24.401345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:24.407867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.414291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23030e-33bc-4444-8ba9-acce5817330f service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.417499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:24.454283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 857/1699] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:24.591320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.591779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.592226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.732885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 843/1700] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:24.787362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.789177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:24.838588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:24.839578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14f00-96ca-4479-8429-481a5b2b52c4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.839982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:24.857890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.858709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.859292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.859618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.881751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.887776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.913434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 844/1701] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:25.083000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 858/1702] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:29:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 26 05:29:25.295394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:25.298899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.307548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:25.309642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.337010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71721a-7059-46d6-be6c-bef2e3c8b868 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.353492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.354307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.354584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.354854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] PATH_INFO: `/v3/users/560af8d93f924c298f7687f249585ce4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.356518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560af8d93f924c298f7687f249585ce4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.359442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.474304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560af8d93f924c298f7687f249585ce4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:25.475107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560af8d93f924c298f7687f249585ce4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:25.487050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560af8d93f924c298f7687f249585ce4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:25.487976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560af8d93f924c298f7687f249585ce4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:25.494645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91017034-e209-4a6d-b600-17fc8fbe0f01 service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.502924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.503493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-604ef93e-70ed-4557-8c16-ee67a2359068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560af8d93f924c298f7687f249585ce4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:25.504130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.504800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.505466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.526635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.529167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.559986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 859/1703] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:25 2022] DELETE /identity/v3/users/560af8d93f924c298f7687f249585ce4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:25.563128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.565409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.567522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.569284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.570694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d455b-988c-4d44-b4f4-09aa67723f6d service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.598678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 845/1704] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:29:25.606680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.606970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.607255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.685654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] MFA Rules not processed for user `683c4d41fa4a452ba8210e62dd43f7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:25.690443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.702857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:25.705578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.926088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 846/1705] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:25.973540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 860/1706] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:26.153455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.159965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:26.188972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:26.354640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad46d8-a6b9-4149-8001-315bb50ab4d8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.366613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.367436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.367732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.368008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.397946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:26.403527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0fde-8a4e-406f-a91c-3b9a122e54cf service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.435944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 847/1707] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:26.684353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.688421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:26.710040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46d52d-b42d-45a3-aff4-44033f6f2583 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.720938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.722075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.722945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.725105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] PATH_INFO: `/v3/projects/e666edf14c434665bec2a5f89efffc8c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.727336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e666edf14c434665bec2a5f89efffc8c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:26.742793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.899558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e666edf14c434665bec2a5f89efffc8c', 'request_id': 'req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:26.932451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e666edf14c434665bec2a5f89efffc8c', 'request_id': 'req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:26.961130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e666edf14c434665bec2a5f89efffc8c', 'request_id': 'req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:27.021008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] The token cache is being invalidate because project e666edf14c434665bec2a5f89efffc8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:27.023593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:27.025802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0c9f426-73b7-4385-ad4a-0cc7401c80e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:27.033913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 861/1708] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:26 2022] DELETE /identity/v3/projects/e666edf14c434665bec2a5f89efffc8c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:27.036020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.044739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.045228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.121395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:27.122872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:27.309706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 848/1709] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:27.311974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.312273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.312646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.362696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:27.370232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:27.392813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 862/1710] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:27.395879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.408062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead022b7-4f8d-445b-b705-c669b13b6b35 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.427568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.428408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.428696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.429048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.429634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:27.429941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.431676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.463464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 849/1711] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:27.468336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.483451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dacd2-ca9e-4143-baa4-9773468ee659 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.492231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.493212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.493727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.494149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.495128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.515608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.607380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 863/1712] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:29:27.611589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.627284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23f81-81a6-435a-8168-1d3743e73409 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.796737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.797505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.797703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.798027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.798702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.806243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.885739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 850/1713] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:27.890185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.907271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8873-9f3a-405f-a267-3ae8bcd2717e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.924282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.925125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.925243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.925512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.926147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.928539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.961990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 864/1714] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:27.966078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.981286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90992450-3554-45e5-a486-5a4eef44b7c7 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.992121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.995544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.995865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.996217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.996990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.998695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.038818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 851/1715] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:29:28.041004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.055144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7604c-d5bc-4cc2-aefc-9ef7b130e689 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.075302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.078699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.081342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.081974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] PATH_INFO: `/v3/projects/21471e53ced347948470159d8ae56189/users/58c65cf2ed3745e8bf17cb4440cd20dc/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.100177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.102214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:28.158480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8447-8467-4d80-aca9-c791c7634962 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.185070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21471e53ced347948470159d8ae56189, user_id=58c65cf2ed3745e8bf17cb4440cd20dc, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.188085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec4fca-87e3-49e5-8ef5-a777f5f40b97 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.194288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.194857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.195136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.195410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] PATH_INFO: `/v3/projects/cce8591f14484fe2ba7ae74c07222b01` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.211986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce8591f14484fe2ba7ae74c07222b01)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.213191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: WARNING py.warnings [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:29:28.213191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: warnings.warn(msg) Sep 26 05:29:28.213191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: Sep 26 05:29:28.214397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.241229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 865/1716] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:28 2022] PUT /identity/v3/projects/21471e53ced347948470159d8ae56189/users/58c65cf2ed3745e8bf17cb4440cd20dc/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:28.243028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.243412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.243657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.288894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.352054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.358020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] MFA Rules not processed for user `58c65cf2ed3745e8bf17cb4440cd20dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:28.371412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:28.542313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce8591f14484fe2ba7ae74c07222b01', 'request_id': 'req-930005f6-e6df-45fa-89d0-478a11cab341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:28.633736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce8591f14484fe2ba7ae74c07222b01', 'request_id': 'req-930005f6-e6df-45fa-89d0-478a11cab341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:28.651772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce8591f14484fe2ba7ae74c07222b01', 'request_id': 'req-930005f6-e6df-45fa-89d0-478a11cab341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:28.688701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] The token cache is being invalidate because project cce8591f14484fe2ba7ae74c07222b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:28.689201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:28.690942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:28.697812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 852/1717] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:28 2022] DELETE /identity/v3/projects/cce8591f14484fe2ba7ae74c07222b01 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:28.697812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.702625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:28.812711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 866/1718] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 571 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:28.817276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.833507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:28.886337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930005f6-e6df-45fa-89d0-478a11cab341 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.911451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.915953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.918666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.921351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] PATH_INFO: `/v3/projects/e1868f47f7ab43d5be20bb69022d51fb` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.925863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1868f47f7ab43d5be20bb69022d51fb)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.930513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.953303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170a0e1-c5b1-46ac-9d36-5abd5047871b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.007016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.007880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.008200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.012717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] PATH_INFO: `/v3/users/5ba6f57d30d4459ea10f852965534866` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.019396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba6f57d30d4459ea10f852965534866)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.022158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.161145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1868f47f7ab43d5be20bb69022d51fb', 'request_id': 'req-00a6eba1-17d9-4541-90f4-113d73baf3b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.188711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1868f47f7ab43d5be20bb69022d51fb', 'request_id': 'req-00a6eba1-17d9-4541-90f4-113d73baf3b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.215555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1868f47f7ab43d5be20bb69022d51fb', 'request_id': 'req-00a6eba1-17d9-4541-90f4-113d73baf3b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.254635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] The token cache is being invalidate because project e1868f47f7ab43d5be20bb69022d51fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:29.254858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.258650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba6f57d30d4459ea10f852965534866'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.259233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.260778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba6f57d30d4459ea10f852965534866'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.267156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 853/1719] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:28 2022] DELETE /identity/v3/projects/e1868f47f7ab43d5be20bb69022d51fb => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:29.276835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.279892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:29.287918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba6f57d30d4459ea10f852965534866'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.289990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba6f57d30d4459ea10f852965534866'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.345859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0ec5595b-c588-4afa-8d22-020a1ba89ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba6f57d30d4459ea10f852965534866'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.439339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:29.448772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 867/1720] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:28 2022] DELETE /identity/v3/users/5ba6f57d30d4459ea10f852965534866 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:29.454567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.454567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.454567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.485632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:29.487240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:29.602191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6eba1-17d9-4541-90f4-113d73baf3b0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.604196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 868/1721] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Sep 26 05:29:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:29.605196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.609885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:29.718727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:29.835018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0091a51-eb94-4ddd-a926-7ca27d7e5b40 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.843653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.844679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.844959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.845228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] PATH_INFO: `/v3/users/429805a9fb4e4c9c950552f5cf07df3e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.846602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=429805a9fb4e4c9c950552f5cf07df3e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.852352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.877408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.878047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.878349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.878602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.882592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:29.988524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429805a9fb4e4c9c950552f5cf07df3e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:29.989426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429805a9fb4e4c9c950552f5cf07df3e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:30.008357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429805a9fb4e4c9c950552f5cf07df3e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:30.009713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429805a9fb4e4c9c950552f5cf07df3e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:30.041667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d46839ad-9e3b-4019-9593-e5c4fa325a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429805a9fb4e4c9c950552f5cf07df3e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:30.094460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:30.096046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:30.111891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 869/1722] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:29 2022] DELETE /identity/v3/users/429805a9fb4e4c9c950552f5cf07df3e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:30.130279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 854/1723] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 862 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:31.191831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294aaeb-26e3-48fd-ad08-974c47e79983 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.192467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294aaeb-26e3-48fd-ad08-974c47e79983 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.192467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294aaeb-26e3-48fd-ad08-974c47e79983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.237818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5294aaeb-26e3-48fd-ad08-974c47e79983 None None] MFA Rules not processed for user `a8f59b47841a4b62b8e368562c7bd210`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:31.239196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5294aaeb-26e3-48fd-ad08-974c47e79983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:31.284501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.343723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a49d91-1d75-4d25-9ae8-723ed7126ac4 service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.431728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 870/1724] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:31.448624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.449260 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.449536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.449807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] PATH_INFO: `/v3/projects/3e4e6b488b75450da9c844cd0499fd34` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.451034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4e6b488b75450da9c844cd0499fd34)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.452928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.462180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.533843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4e6b488b75450da9c844cd0499fd34', 'request_id': 'req-1648b7d9-c37c-451c-b353-9eda226ca6ea'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:31.550399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4e6b488b75450da9c844cd0499fd34', 'request_id': 'req-1648b7d9-c37c-451c-b353-9eda226ca6ea'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:31.563770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4e6b488b75450da9c844cd0499fd34', 'request_id': 'req-1648b7d9-c37c-451c-b353-9eda226ca6ea'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:31.579384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] The token cache is being invalidate because project 3e4e6b488b75450da9c844cd0499fd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:31.579384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:31.580672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1648b7d9-c37c-451c-b353-9eda226ca6ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:31.583084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 855/1725] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:31 2022] DELETE /identity/v3/projects/3e4e6b488b75450da9c844cd0499fd34 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:31.587988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.588439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.588782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.611431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:31.612886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:31.691003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.719127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 871/1726] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:31.727693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.728021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.728268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.753759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:31.755395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:31.769865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 856/1727] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:31.771272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.784544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec06c8-0526-48dc-b722-2476ff6436cb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.798514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.800164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:31.800308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.801181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.801523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.801813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.804049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:31.804049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.805654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.819701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 872/1728] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:31.822047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2bcb2-a909-41d9-97b0-6775bb14f6bd None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.823753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.831872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.832656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.833280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.833651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] PATH_INFO: `/v3/projects/0a2db628604842f1bcec89c78aca3b43` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.835279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a999-a620-4fe4-9fe1-684582eee5ef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.835465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2db628604842f1bcec89c78aca3b43)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.838112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.857445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.858300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.859145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.859593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.860687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.863372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.956032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 873/1729] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:31.965474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.993474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2db628604842f1bcec89c78aca3b43', 'request_id': 'req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:32.010660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2db628604842f1bcec89c78aca3b43', 'request_id': 'req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:32.026718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2db628604842f1bcec89c78aca3b43', 'request_id': 'req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:32.046596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] The token cache is being invalidate because project 0a2db628604842f1bcec89c78aca3b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:32.047381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:32.048306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0dcb40ea-bd0b-46f3-95e8-d07b4f36388e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:32.052298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 857/1730] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:31 2022] DELETE /identity/v3/projects/0a2db628604842f1bcec89c78aca3b43 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:32.056623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.056623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.056924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.123303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:32.124706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.209787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5c11-3f09-46ff-8420-b749344a0ec8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.216750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.217691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.218182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.218685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.219548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.223212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.293425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 874/1731] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/users => generated 278 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:32.300772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.304992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.378007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca609-7b66-4504-8dd5-d4bff7bde46c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.392947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 858/1732] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:32.393935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.409931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.418111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.427524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.429898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.431117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.432404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.444382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3980f4-500a-473b-8a29-3b1f160c54cd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.471122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:32.490779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 875/1733] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:32.494944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.495178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.497601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.596600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:32.598357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.672759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 876/1734] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:32.672759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.702022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd20294-5c1b-4933-9e61-ad5cb72481ee None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.713609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fb2c9-e747-4db1-99d9-39f0143bef76 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.733940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.737021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.745142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.745142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.745142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.745426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.745645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.745784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.745784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.745784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.789780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.793684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.811089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 877/1735] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:32.815151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.850920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 859/1736] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:32.851996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.863641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb1d82-d5f8-4534-a7fb-b702077bfa61 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.899933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.900872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.901273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.905941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.905941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:32.906328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.908121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.914067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d01f-c4b2-4644-9306-12fb790e62eb service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.941834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.942703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.953302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.953664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] PATH_INFO: `/v3/projects/e56f164b9ad6411ca4455f692c131d1c/users/450b7ef5a2fc434d98ae5c887fa89328/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.962593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 878/1737] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:32.967470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.998354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53921f0-3a79-4e84-9eb2-5fe20386bf39 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.023690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.034076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.034286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56f164b9ad6411ca4455f692c131d1c, user_id=450b7ef5a2fc434d98ae5c887fa89328, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.035296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.035756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.039151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.049605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221243ae-0bb9-4aba-9ac3-113e9fb3a3d5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.050023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.098772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 860/1738] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:32 2022] PUT /identity/v3/projects/e56f164b9ad6411ca4455f692c131d1c/users/450b7ef5a2fc434d98ae5c887fa89328/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:33.120863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.121185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.121504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.179399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 879/1739] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:29:33.184599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.258760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] MFA Rules not processed for user `450b7ef5a2fc434d98ae5c887fa89328`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:33.260105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:33.404020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:33.498571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 861/1740] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:33.502805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.510619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dc15a-4d47-40a6-a332-667f2b00335b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.523252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.524170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.524487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.524767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.525416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.535180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993d84-f714-41ad-9944-2d666cde43eb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.547806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.548276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.548553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.548823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.549487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.551167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.551746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.645541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 862/1741] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:33 2022] POST /identity/v3/projects => generated 357 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:29:33.645937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 880/1742] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:33 2022] POST /identity/v3/users => generated 290 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:33.646542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.649882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.753524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882166e-2d2d-4e68-8342-c292cf5decb7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.860459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633d80e-bfbe-4c63-a376-3fb907547b22 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.881071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.881954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.882374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.882770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.884512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.886262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.901393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:33.931537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 863/1743] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:33.933354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.967449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f1179-42a2-4e5e-8f2c-db4c68a4b0df admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.992440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.992744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.993155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.993280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.993896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.995524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.057234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 864/1744] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:34.058627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.093976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312b9e-b6c9-4778-ab79-e1c50cdf5e12 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:34.116598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.117338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.121244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.121572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] PATH_INFO: `/v3/projects/4ac5bd4c76944b26b4d476a04fef76db/users/4c058ab9d1d542ea9d26f2bf2e1150cd/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.171137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac5bd4c76944b26b4d476a04fef76db, user_id=4c058ab9d1d542ea9d26f2bf2e1150cd, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.173574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e7e9-505c-4928-b848-9e2458ca7d6b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.181684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.185753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.188619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.189018 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.190987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.206596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa49d9-a396-4a3d-8a3f-fc667512d1f8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.221576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 865/1745] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:34 2022] PUT /identity/v3/projects/4ac5bd4c76944b26b4d476a04fef76db/users/4c058ab9d1d542ea9d26f2bf2e1150cd/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:34.223281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.223567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.224030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.289169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] MFA Rules not processed for user `48e588ff6d5a4a37b902abadfb6d04fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:34.290434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:34.308591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 881/1746] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:33 2022] POST /identity/v3/users => generated 285 bytes in 659 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:34.310832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.326114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.335183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.438499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] MFA Rules not processed for user `4c058ab9d1d542ea9d26f2bf2e1150cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:34.439768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:34.621088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 866/1747] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:34.625197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.673154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 882/1748] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:34.674381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.873779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:34.898201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa87248-a526-4fc3-ae22-371f700539ce None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:34.912591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.913544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.914355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.914794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.915770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.921385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.975298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 883/1749] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:34 2022] POST /identity/v3/projects => generated 379 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:34.980539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.982029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0089b0-c9d7-4ee7-903a-4d10926ed29d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.196425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f7d3-8a85-47b5-a857-3e51d494b695 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.204162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.204935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.205320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.205518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.205796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.206145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.206428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.206703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.207542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.208377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.209793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.213469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.247776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 867/1750] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 624 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:35.249140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:35.274703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef63ddd-2f08-4298-92fc-538e434d93e3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.288112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.288637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 884/1751] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:34 2022] POST /identity/v3/users => generated 290 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:35.289419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.289810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.290657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] PATH_INFO: `/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d/users/e3159295577d483b8fb3230e60dbdfce/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.303263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:35.310207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:35.359841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274a17d-49a8-4738-a8c4-8236c4f22701 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.364087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1ef3199e47410ba151cd7edcb4fd5d, user_id=e3159295577d483b8fb3230e60dbdfce, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.368185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.389615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.391893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.392170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.396900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.401377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.410537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.416700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 868/1752] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:35 2022] PUT /identity/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d/users/e3159295577d483b8fb3230e60dbdfce/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:35.419216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:35.451763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 885/1753] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:35.460117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:35.498510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2c4c-5ba1-4d37-b788-3cd380c91713 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.581814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9de588-80d5-4dcc-a884-98dcbca89ce7 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.593448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.595118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.595561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.596776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.599057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.605302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.631401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 869/1754] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:35.638525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:35.658921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c50b7-a324-45d3-b9de-5d780d0ec081 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:35.672152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.672874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.674129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.674129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] PATH_INFO: `/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d/users/e3159295577d483b8fb3230e60dbdfce/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.677914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1ef3199e47410ba151cd7edcb4fd5d, user_id=e3159295577d483b8fb3230e60dbdfce, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.686298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c027c6-1b89-404e-ad24-0cf8baa10cab admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:35.718390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 870/1755] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:35 2022] PUT /identity/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d/users/e3159295577d483b8fb3230e60dbdfce/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:35.730508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.731102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.731526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.813681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] MFA Rules not processed for user `e3159295577d483b8fb3230e60dbdfce`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:35.818965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:35.917053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:35.919353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:35.919763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:35.920205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:35.922318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:35.937116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:36.029005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 886/1756] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 571 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:36.039624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:36.104500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 871/1757] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:36.104500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:36.111462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:36.115481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59534e51-1081-4098-86d5-aabfe74fa1f3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:36.238693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:36.275210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:36.275988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:36.276280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:36.276834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] PATH_INFO: `/v3/projects/3c1c1542b6b54f8b8bcfafd2b3acd0ed/users/048ca18dd766499a83c5489cb0e30ff7/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:36.344286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1c1542b6b54f8b8bcfafd2b3acd0ed, user_id=048ca18dd766499a83c5489cb0e30ff7, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:36.370834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:36.402515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 887/1758] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:36 2022] PUT /identity/v3/projects/3c1c1542b6b54f8b8bcfafd2b3acd0ed/users/048ca18dd766499a83c5489cb0e30ff7/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:36.404803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:36.414196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:36.416353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:36.464877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001d427-c814-4dba-a319-3f40ce0cbd47 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:36.691277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:36.694444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:36.703293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:36.703921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] PATH_INFO: `/v3/users/58c65cf2ed3745e8bf17cb4440cd20dc` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:36.710034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c65cf2ed3745e8bf17cb4440cd20dc)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:36.710407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4d71-567c-46aa-8fac-1fe0cf4bfe35 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:36.713462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:36.722197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:36.723507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:36.723783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:36.727297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:36.935488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:36.939888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4d9cc-cc4e-49d7-a133-07db924d7d21 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:36.954968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c65cf2ed3745e8bf17cb4440cd20dc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:36.973010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 888/1759] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:36.982086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:36.982694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:36.984396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:36.994106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c65cf2ed3745e8bf17cb4440cd20dc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:36.997126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c65cf2ed3745e8bf17cb4440cd20dc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:37.024076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c65cf2ed3745e8bf17cb4440cd20dc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:37.026562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3eec2ef-eb1e-4ff1-94b6-ada3e3aba5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c65cf2ed3745e8bf17cb4440cd20dc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:37.088183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 872/1760] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:36 2022] DELETE /identity/v3/users/58c65cf2ed3745e8bf17cb4440cd20dc => generated 0 bytes in 986 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:37.109358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.113710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.114342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.139820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] MFA Rules not processed for user `048ca18dd766499a83c5489cb0e30ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:37.141050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.169542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:37.171355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.421709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 873/1761] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:37.428937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:37.438529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.447999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 889/1762] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:37.449405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:37.457523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.492276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:37.582558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:37.698074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12640a9-bb32-4a2d-b7ac-33d1b796837c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:37.716154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:37.716914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.717188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.717473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.766230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924910d-2b95-48cc-a354-102e08aff0e0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:37.769564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:37.771201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:37.789857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:37.790514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.790795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.791068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.803900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.833897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 890/1763] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:37.836950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:37.854331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.916742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309c25-2685-4f28-b8c9-e757b98d3f22 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:37.953852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:37.954471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.954745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.957401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.959177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:37.967918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.056594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 891/1764] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:37 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:38.061193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.077046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:38.078959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.103267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 874/1765] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:38.386706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab74a6-a6f9-47a3-a93d-e875f193fe0f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.417196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.417848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.418160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.418493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.419159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:38.421266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.546719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 892/1766] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:38 2022] POST /identity/v3/users => generated 295 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:38.548128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.565438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74036-6b2a-4d45-b201-3186527964b2 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.589063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.592052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.593994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.595524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.597006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:38.605601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.663211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 875/1767] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:38.663527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.678650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5f26-5461-45d5-9bea-3eadf715a7fb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.711014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.715546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.716336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.717778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] PATH_INFO: `/v3/projects/a0c9eed5a3d749df94bc6bab897068d2/users/fec6d8e903a6497aabcd6fc345f9dbdf/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.801010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c9eed5a3d749df94bc6bab897068d2, user_id=fec6d8e903a6497aabcd6fc345f9dbdf, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:38.805209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.839277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 893/1768] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:38 2022] PUT /identity/v3/projects/a0c9eed5a3d749df94bc6bab897068d2/users/fec6d8e903a6497aabcd6fc345f9dbdf/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:38.843434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.861102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bda10-cdd3-46e9-8938-ff4879c1a42a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.173766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.175737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.176975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.179693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.181580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.185881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.226250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.227041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 876/1769] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:39.271439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fcaa6-0a85-4015-9972-901c8d813cb6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.295503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.297912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.298639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.298923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] PATH_INFO: `/v3/projects/a0c9eed5a3d749df94bc6bab897068d2/users/fec6d8e903a6497aabcd6fc345f9dbdf/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.308489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.310764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c9eed5a3d749df94bc6bab897068d2, user_id=fec6d8e903a6497aabcd6fc345f9dbdf, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.311788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:39.331218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e6572-4b9b-4ef4-89ff-07b2565a90bd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.345553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82650771-0df7-4eb0-a44f-388d446a3b57 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.368665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.369342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.369630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.369913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] PATH_INFO: `/v3/projects/21471e53ced347948470159d8ae56189` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.374750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 894/1770] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:39 2022] PUT /identity/v3/projects/a0c9eed5a3d749df94bc6bab897068d2/users/fec6d8e903a6497aabcd6fc345f9dbdf/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:39.385667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.385988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.386250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.388083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=21471e53ced347948470159d8ae56189)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.395648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.510735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] MFA Rules not processed for user `fec6d8e903a6497aabcd6fc345f9dbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:39.512105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:39.602185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21471e53ced347948470159d8ae56189', 'request_id': 'req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:39.629362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.637119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21471e53ced347948470159d8ae56189', 'request_id': 'req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:39.645427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.664596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21471e53ced347948470159d8ae56189', 'request_id': 'req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:39.706338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] The token cache is being invalidate because project 21471e53ced347948470159d8ae56189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:39.706500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:39.719068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dbf2e2cc-7cc4-4f61-b01a-a2da8d7912ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:39.727191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 877/1771] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:39 2022] DELETE /identity/v3/projects/21471e53ced347948470159d8ae56189 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:39.733142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9939-cfba-48b0-af80-5e43a683cca1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.734439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9939-cfba-48b0-af80-5e43a683cca1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.734439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9939-cfba-48b0-af80-5e43a683cca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.747257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 895/1772] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:39.781405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e3da9939-cfba-48b0-af80-5e43a683cca1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:39.783474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e3da9939-cfba-48b0-af80-5e43a683cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:39.838140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.839791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:39.991905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 878/1773] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:39.997876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.999218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.999742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.019825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b07338-a970-4f20-9be9-0a4bf764a669 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.077962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:40.086409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.133942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 879/1774] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:40.137919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.149258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9c4a-612d-48f7-8562-83fe9b259c54 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.188071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.188815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.189075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.189348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.190001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:40.190303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.191955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.218281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.218986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.219329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.219611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.224807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.244387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 880/1775] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:40.245099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.275743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e2485-405b-4666-91a8-bb4be676096a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.293146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.293772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.294053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.294322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.294971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.296942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.406878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 881/1776] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:40 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:40.407997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.430049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.436886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.500553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 896/1777] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:40.585999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.588539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.620427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8838d-d61f-4240-9a3a-5095a7723659 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.622556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38b7af-add6-493e-a02b-2389124619d3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.645314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.647442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.647912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.648366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.649190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.651987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.714343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.725957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 882/1778] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:40 2022] POST /identity/v3/users => generated 284 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:40.731228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.772285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.772285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.772285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.772285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.780297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.784675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f24864-2fda-4f1f-80ec-d46c52182109 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.814824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.815559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.816910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.822052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.823841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.826584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.859613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 883/1779] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:40.864115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.914538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.922996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.936981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 897/1780] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:40.942589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f606-d5e4-434e-992d-f4bda2793f84 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.978919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.979551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.979829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.980104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.980783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.993167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76e0fb-fea0-4211-ad6c-579e774a2176 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.037491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 884/1781] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:41.041890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.070375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f871-a85a-4885-b949-63c6f114bacc service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.088325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.089040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.094476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.094842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] PATH_INFO: `/v3/projects/02cbffc3f75e4b57b6ebc934d27aa8f3/users/033e6ff5e8f847918f85e0e31d6e4123/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.168749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02cbffc3f75e4b57b6ebc934d27aa8f3, user_id=033e6ff5e8f847918f85e0e31d6e4123, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.173269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.238523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 898/1782] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:41 2022] PUT /identity/v3/projects/02cbffc3f75e4b57b6ebc934d27aa8f3/users/033e6ff5e8f847918f85e0e31d6e4123/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:41.250474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c8fd-934e-4fd1-b71e-37be0f4c3376 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.251079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c8fd-934e-4fd1-b71e-37be0f4c3376 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.251451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c8fd-934e-4fd1-b71e-37be0f4c3376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.327241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.350923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-26a5c8fd-934e-4fd1-b71e-37be0f4c3376 None None] MFA Rules not processed for user `033e6ff5e8f847918f85e0e31d6e4123`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:41.356131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-26a5c8fd-934e-4fd1-b71e-37be0f4c3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:41.546942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 885/1783] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:41.700716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.703766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:41.918145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff956480-3185-4d4b-88e4-a156333486a8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.941623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.946081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.949467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.950571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.004229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.010097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d886230-6aa8-4a03-89cd-1318d43bba17 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.075313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 899/1784] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:42.582377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.584154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.584824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.587059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 886/1785] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:29:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:42.934562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.935189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.935189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.997476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:42.999574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:43.175464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:43.217548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 900/1786] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:43.298056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:43.299886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:43.369378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58998806-0a77-4ff8-bdb9-8860677ccaaa None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:43.387483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.401892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.402194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.402508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.434963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.438395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ed911-cb37-449a-8798-447d240c7cb6 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.466825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 887/1787] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:45.260872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:45.264526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:45.406455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18532d-471c-4ffc-be85-d19da824073e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:45.415205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:45.416188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:45.416520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:45.416889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] PATH_INFO: `/v3/users/4c058ab9d1d542ea9d26f2bf2e1150cd` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:45.417994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c058ab9d1d542ea9d26f2bf2e1150cd)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:45.420190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:45.487815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c058ab9d1d542ea9d26f2bf2e1150cd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:45.489082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c058ab9d1d542ea9d26f2bf2e1150cd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:45.502827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c058ab9d1d542ea9d26f2bf2e1150cd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:45.503820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c058ab9d1d542ea9d26f2bf2e1150cd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:45.522163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c058ab9d1d542ea9d26f2bf2e1150cd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:45.591767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 901/1788] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:45 2022] DELETE /identity/v3/users/4c058ab9d1d542ea9d26f2bf2e1150cd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:45.593514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109e78-78fe-4ee2-afed-0a58dec41e90 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:45.594029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109e78-78fe-4ee2-afed-0a58dec41e90 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:45.594285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109e78-78fe-4ee2-afed-0a58dec41e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:45.625649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-27109e78-78fe-4ee2-afed-0a58dec41e90 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:45.633156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-27109e78-78fe-4ee2-afed-0a58dec41e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:45.822685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 888/1789] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:45.852055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:45.853973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:45.986669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f5eb0-7c66-4e91-83f4-a78036e64d12 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:45.991819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:45.992692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:45.993116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:45.993391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.015639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:46.017230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a667703-7b64-4a23-a569-2377ccf7a890 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:46.033612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 902/1790] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:46.046572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.046867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.047147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.049081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 889/1791] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:46.107231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.107712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.108231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.161870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:46.163595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:46.200653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:46.354030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 903/1792] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:46.727399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:46.729471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:46.782296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1016e2-661f-47d4-89ac-f3eccd6f800d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:46.790958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:46.791965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.792456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.793066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.807317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:46.810849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:46.827808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 890/1793] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:46.875644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:46.879096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:46.918833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3285d10-7b9f-4a59-bab4-59e813fc134d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:46.938335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:46.939554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.939904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.940279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] PATH_INFO: `/v3/projects/4ac5bd4c76944b26b4d476a04fef76db` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.941550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac5bd4c76944b26b4d476a04fef76db)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:46.944145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:47.014539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac5bd4c76944b26b4d476a04fef76db', 'request_id': 'req-f313938d-95cd-4148-9c9f-7e308dbb7f25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.030296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac5bd4c76944b26b4d476a04fef76db', 'request_id': 'req-f313938d-95cd-4148-9c9f-7e308dbb7f25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.044487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac5bd4c76944b26b4d476a04fef76db', 'request_id': 'req-f313938d-95cd-4148-9c9f-7e308dbb7f25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.070287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] The token cache is being invalidate because project 4ac5bd4c76944b26b4d476a04fef76db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:47.070743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.071797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.073498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 904/1794] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:46 2022] DELETE /identity/v3/projects/4ac5bd4c76944b26b4d476a04fef76db => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:47.078393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:47.100900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8888e3-3e12-4788-88a3-54a39b5e233a service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:47.220039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:47.220826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.230687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.231136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] PATH_INFO: `/v3/users/048ca18dd766499a83c5489cb0e30ff7` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.233426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=048ca18dd766499a83c5489cb0e30ff7)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:47.236057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:47.293022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:47.378544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048ca18dd766499a83c5489cb0e30ff7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.395653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048ca18dd766499a83c5489cb0e30ff7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.397096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048ca18dd766499a83c5489cb0e30ff7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.412174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048ca18dd766499a83c5489cb0e30ff7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.413142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048ca18dd766499a83c5489cb0e30ff7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:47.461714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 891/1795] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:47 2022] DELETE /identity/v3/users/048ca18dd766499a83c5489cb0e30ff7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:48.290938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.292516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:48.314554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313938d-95cd-4148-9c9f-7e308dbb7f25 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.387239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.388112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.388421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.388765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] PATH_INFO: `/v3/projects/3c1c1542b6b54f8b8bcfafd2b3acd0ed` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.389965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1c1542b6b54f8b8bcfafd2b3acd0ed)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.391889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.448237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.457059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1c1542b6b54f8b8bcfafd2b3acd0ed', 'request_id': 'req-4add2b46-e967-4ec9-9272-288e768051fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.470614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1c1542b6b54f8b8bcfafd2b3acd0ed', 'request_id': 'req-4add2b46-e967-4ec9-9272-288e768051fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.477256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1c1542b6b54f8b8bcfafd2b3acd0ed', 'request_id': 'req-4add2b46-e967-4ec9-9272-288e768051fe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.491176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] The token cache is being invalidate because project 3c1c1542b6b54f8b8bcfafd2b3acd0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:48.491531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.492272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.494003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 905/1796] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:48 2022] DELETE /identity/v3/projects/3c1c1542b6b54f8b8bcfafd2b3acd0ed => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:48.498085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.499556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:48.512826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44462234-721b-498c-944d-5705535fbcd7 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.590128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.590765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.591038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.591333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] PATH_INFO: `/v3/users/fec6d8e903a6497aabcd6fc345f9dbdf` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.592531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec6d8e903a6497aabcd6fc345f9dbdf)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.594568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.671149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec6d8e903a6497aabcd6fc345f9dbdf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.699935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec6d8e903a6497aabcd6fc345f9dbdf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.701337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec6d8e903a6497aabcd6fc345f9dbdf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.721039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec6d8e903a6497aabcd6fc345f9dbdf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.723396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-778f4a44-8aeb-48c9-9972-2f214c13e1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec6d8e903a6497aabcd6fc345f9dbdf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:48.758854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 892/1797] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:48 2022] DELETE /identity/v3/users/fec6d8e903a6497aabcd6fc345f9dbdf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:50.367833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.371099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:50.388539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2b46-e967-4ec9-9272-288e768051fe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.447806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97287-d238-4070-9c7d-385caf916949 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.448381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.448915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.449189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.449455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] PATH_INFO: `/v3/projects/a0c9eed5a3d749df94bc6bab897068d2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.449627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97287-d238-4070-9c7d-385caf916949 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.450048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97287-d238-4070-9c7d-385caf916949 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.451039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c9eed5a3d749df94bc6bab897068d2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.451861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 893/1798] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:29:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:50.453267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.482368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.484485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.485255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.514662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c9eed5a3d749df94bc6bab897068d2', 'request_id': 'req-cf84bf36-80ba-431c-ae95-f0504e78608e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:50.519319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:50.520901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:50.527438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c9eed5a3d749df94bc6bab897068d2', 'request_id': 'req-cf84bf36-80ba-431c-ae95-f0504e78608e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:50.528240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:50.535054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c9eed5a3d749df94bc6bab897068d2', 'request_id': 'req-cf84bf36-80ba-431c-ae95-f0504e78608e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:50.557394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] The token cache is being invalidate because project a0c9eed5a3d749df94bc6bab897068d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:29:50.557394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:50.557855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf84bf36-80ba-431c-ae95-f0504e78608e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:50.560297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 906/1799] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:50 2022] DELETE /identity/v3/projects/a0c9eed5a3d749df94bc6bab897068d2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:50.566918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.567564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.568155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.605414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:50.608156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:50.614744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 894/1800] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:50.629953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.632147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:50.650457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de20a86-f0ef-4e34-864f-1d25b5c91be7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.656850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.657459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.657736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.658007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.667958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.669599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b961b9-58ba-41e1-b974-dd644c9922da service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.678156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:50.682257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 895/1801] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:29:50.696645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 907/1802] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:50.700332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.700731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.701070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.722403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:50.724333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:50.732218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 896/1803] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:50.736945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.747269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0844b-2d5c-47a4-91ab-f44bf0e9e02d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.752033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.752729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.753010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.753369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.753913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:50.754214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.756331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.772812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 908/1804] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:50.777201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.788858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cd078-37c1-43d1-a0b2-17c9d6417ec6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.793582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.794191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.794466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.794754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.795373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.797318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.835287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 897/1805] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:29:50 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:50.839351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.850379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b076-8a0c-4f29-bf69-b4853291def2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.975988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.977141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.978573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.979037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.981322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.991753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.061495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:51.077568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2b37e-df16-43b4-b101-d11c2428c187 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:51.078053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 909/1806] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:29:50 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:51.103566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:51.104192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.112029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.112926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.113155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:51.114852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.145218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 898/1807] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:51.148579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:51.163941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537c870-1333-4101-a0ea-8049e3647bf9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:51.171268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:51.177668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.179761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.180916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.184052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:51.186248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12430e-57f5-444b-bd49-2be0b9041405 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.201590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 910/1808] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:29:51.205479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:51.218607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813f84f-ae71-41bf-9f24-569783e876d9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:51.225334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:51.226393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.226932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.227380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] PATH_INFO: `/v3/projects/a6284cd6ccab4999af9dc5946e7c9722/users/f5af5d57d5cd4f76a11116f26c50716e/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.245875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6284cd6ccab4999af9dc5946e7c9722, user_id=f5af5d57d5cd4f76a11116f26c50716e, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:51.250570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.278138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 899/1809] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:29:51 2022] PUT /identity/v3/projects/a6284cd6ccab4999af9dc5946e7c9722/users/f5af5d57d5cd4f76a11116f26c50716e/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:51.291313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.292000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.292514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.349435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] MFA Rules not processed for user `f5af5d57d5cd4f76a11116f26c50716e`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:51.350672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:51.491459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 911/1810] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:51.503461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:51.505877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:51.604504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5a42e-fa67-47e3-b44d-17336d8ed165 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:51.608137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:51.608898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.609245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.609579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.620343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:51.621808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc48f-f570-47d3-817c-1d1f7b037c39 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.631725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 900/1811] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:58.928008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:58.998061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:59.031584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0d081-5a43-4fca-a5f1-af667a811199 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:59.038100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:59.038815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.039112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.039477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] PATH_INFO: `/v3/users/f5af5d57d5cd4f76a11116f26c50716e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.041281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5af5d57d5cd4f76a11116f26c50716e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:59.043224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:59.118505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5af5d57d5cd4f76a11116f26c50716e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.119201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5af5d57d5cd4f76a11116f26c50716e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.133750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5af5d57d5cd4f76a11116f26c50716e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.135165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5af5d57d5cd4f76a11116f26c50716e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.148188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5af5d57d5cd4f76a11116f26c50716e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.180603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 912/1812] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:58 2022] DELETE /identity/v3/users/f5af5d57d5cd4f76a11116f26c50716e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:59.181622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.181622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.181622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.204400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:59.206205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.210391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:59.212188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.247392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:59.300396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881c21-46ed-4000-b9c8-11190224a412 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:59.307195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:59.308351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.308877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.309570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] PATH_INFO: `/v3/users/033e6ff5e8f847918f85e0e31d6e4123` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.311189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] RBAC: Authorizing `identity:delete_user(user_id=033e6ff5e8f847918f85e0e31d6e4123)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:59.313932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:59.356563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 901/1813] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:29:59.360189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:59.363321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.398417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '033e6ff5e8f847918f85e0e31d6e4123'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.399179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '033e6ff5e8f847918f85e0e31d6e4123'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.408455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '033e6ff5e8f847918f85e0e31d6e4123'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.409212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '033e6ff5e8f847918f85e0e31d6e4123'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.419285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-19e0b721-99f8-407c-a73b-3c2e214f253d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '033e6ff5e8f847918f85e0e31d6e4123'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.428798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:59.448474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 913/1814] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:59 2022] DELETE /identity/v3/users/033e6ff5e8f847918f85e0e31d6e4123 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:59.452764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.453158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.453516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.458905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352528e0-cb04-4482-83e6-685e9c6f4905 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:59.478381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:59.479593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.513181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:59.513711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.513986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.514256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.516096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.557806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 914/1815] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:29:59.572664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:59.574734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.594442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:59.596804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:59.607649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131620-1ddc-40d9-9e40-a914c9c88813 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:59.612371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:59.612656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 902/1816] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:59.612987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.613393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.615579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.630035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:59.632118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9532e0-e866-4227-a447-58b6da906529 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:59.643961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 915/1817] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:29:59.661300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:59.663492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.677565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26858-68c6-4eec-83bc-cacbb91a03f7 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:59.684029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:59.685446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.685975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.686420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] PATH_INFO: `/v3/users/450b7ef5a2fc434d98ae5c887fa89328` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.688422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] RBAC: Authorizing `identity:delete_user(user_id=450b7ef5a2fc434d98ae5c887fa89328)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:59.690899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:59.750697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:59.771898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450b7ef5a2fc434d98ae5c887fa89328'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.783726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450b7ef5a2fc434d98ae5c887fa89328'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.785148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450b7ef5a2fc434d98ae5c887fa89328'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.805118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450b7ef5a2fc434d98ae5c887fa89328'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.819655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450b7ef5a2fc434d98ae5c887fa89328'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:29:59.862146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 903/1818] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:29:59 2022] DELETE /identity/v3/users/450b7ef5a2fc434d98ae5c887fa89328 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:59.864803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:59.865527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:59.866017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:59.893563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:59.895518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:59.972160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:59.974955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.027720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 916/1819] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:00.042884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.044466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.117685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30c193-0518-40be-a054-0e2058d4194f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:00.128995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:00.130262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.130553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.130826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] PATH_INFO: `/v3/projects/a6284cd6ccab4999af9dc5946e7c9722` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.132088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6284cd6ccab4999af9dc5946e7c9722)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:00.134072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:00.173638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7eb8-fe3d-46b1-9c3c-7b34eeedc571 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:00.183449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:00.183757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.184979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.185569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.206468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:00.210383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:00.242446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6284cd6ccab4999af9dc5946e7c9722', 'request_id': 'req-dab58f83-a538-4407-a931-b05033773fa5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.247547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 917/1820] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:00.248105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.253218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.255526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6284cd6ccab4999af9dc5946e7c9722', 'request_id': 'req-dab58f83-a538-4407-a931-b05033773fa5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.273046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6284cd6ccab4999af9dc5946e7c9722', 'request_id': 'req-dab58f83-a538-4407-a931-b05033773fa5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.283214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6709f-795f-430d-9c6c-55650c06ecb9 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:00.289242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] The token cache is being invalidate because project a6284cd6ccab4999af9dc5946e7c9722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:00.289538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.301798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dab58f83-a538-4407-a931-b05033773fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.304642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 904/1821] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:29:59 2022] DELETE /identity/v3/projects/a6284cd6ccab4999af9dc5946e7c9722 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:00.310821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01efa6-b4e4-4355-8e27-240ecd8f4453 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.311248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01efa6-b4e4-4355-8e27-240ecd8f4453 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.311577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01efa6-b4e4-4355-8e27-240ecd8f4453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.326383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:00.357616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-cd01efa6-b4e4-4355-8e27-240ecd8f4453 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.358861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cd01efa6-b4e4-4355-8e27-240ecd8f4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.421850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:00.422563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.422850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.423240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] PATH_INFO: `/v3/projects/02cbffc3f75e4b57b6ebc934d27aa8f3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.428778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=02cbffc3f75e4b57b6ebc934d27aa8f3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:00.430998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:00.439820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:00.506483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 905/1822] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:00.516073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.516445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.516852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.561723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.562964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.574127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02cbffc3f75e4b57b6ebc934d27aa8f3', 'request_id': 'req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.598813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02cbffc3f75e4b57b6ebc934d27aa8f3', 'request_id': 'req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.607204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02cbffc3f75e4b57b6ebc934d27aa8f3', 'request_id': 'req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.631692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] The token cache is being invalidate because project 02cbffc3f75e4b57b6ebc934d27aa8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:00.632003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.632824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9b7946e7-834f-40dd-bf74-8d0ee5827b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:00.634673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 918/1823] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:00 2022] DELETE /identity/v3/projects/02cbffc3f75e4b57b6ebc934d27aa8f3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:00.639839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fbef-e9bf-4156-b2c8-b897e2277b3e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.640169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fbef-e9bf-4156-b2c8-b897e2277b3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.640581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fbef-e9bf-4156-b2c8-b897e2277b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.683483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8cb5fbef-e9bf-4156-b2c8-b897e2277b3e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.696075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8cb5fbef-e9bf-4156-b2c8-b897e2277b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.742954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 906/1824] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:00.747483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.767183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c5432-4062-4d10-a31c-2b8f9fed9b16 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:00.775715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:00.776212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.776629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.776905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.777731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:00.778048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:00.779670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:00.806033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 907/1825] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:00.810972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.837642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 919/1826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:00.837862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049526-c231-47e8-a9f7-b679f9ce1471 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:00.839429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.839722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.839997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.853927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:00.854815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.855309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.855769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.856678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:00.858969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:00.888885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.890307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.901732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 920/1827] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:00.913573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.939001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 908/1828] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:00.942561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:00.959603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592ef6d-3fbd-45e5-8fa5-4b11685001ea None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.024534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.044704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.045349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.045589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.045855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.046541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:01.046844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e8fad-7e27-4a39-8baa-032821b637ae admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.046969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.048550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.052479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.053397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.053893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.054311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.055259 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.057759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.062334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 921/1829] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:01.068081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.087959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 909/1830] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:01.088928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.090441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ac13c-312b-4a62-aa3d-087e09752db1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.090650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.095514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.096124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.096459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.096890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.097747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.101081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.103958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501989-238a-4f94-9c41-7460e72ac346 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.117887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.118547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.118831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.119108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] PATH_INFO: `/v3/projects/e56f164b9ad6411ca4455f692c131d1c` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.120337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56f164b9ad6411ca4455f692c131d1c)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.122319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.136305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 922/1831] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:01.136305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.158447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.177676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56f164b9ad6411ca4455f692c131d1c', 'request_id': 'req-69ab15ef-9117-4dbd-95d3-5a001f77869c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.186061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56f164b9ad6411ca4455f692c131d1c', 'request_id': 'req-69ab15ef-9117-4dbd-95d3-5a001f77869c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.194338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56f164b9ad6411ca4455f692c131d1c', 'request_id': 'req-69ab15ef-9117-4dbd-95d3-5a001f77869c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.206201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d35d2-121b-4d06-be23-50f0f66dd118 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.206896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] The token cache is being invalidate because project e56f164b9ad6411ca4455f692c131d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:01.207218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.208011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.209804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 910/1832] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:01 2022] DELETE /identity/v3/projects/e56f164b9ad6411ca4455f692c131d1c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:01.210381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.211678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.276410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.277146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.277430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.277760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.278457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.280121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.283346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab15ef-9117-4dbd-95d3-5a001f77869c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.288187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.288813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.289088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.289355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.289977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.293381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 923/1833] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:01.293381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.293547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.293620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.307520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fb80-43e7-47e7-a713-77cd38333e6d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.312616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.313199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.313497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.313753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] PATH_INFO: `/v3/users/e3159295577d483b8fb3230e60dbdfce` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.314887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3159295577d483b8fb3230e60dbdfce)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.320162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 911/1834] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:01.320424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.320516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.320516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.334458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c4-f8d2-4425-94ab-097f91fe5601 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.341600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.342349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.342478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.342748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.343387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.345345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.357170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 912/1835] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:01.358117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.374720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca76b20-1bb7-4f62-b72b-4ea712b3f4d9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.380474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3159295577d483b8fb3230e60dbdfce'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.381508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3159295577d483b8fb3230e60dbdfce'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.382606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.383368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.383707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.384050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.384816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.386499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.392271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3159295577d483b8fb3230e60dbdfce'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.393665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3159295577d483b8fb3230e60dbdfce'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.395902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.402079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 913/1836] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:01.403343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.403739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3159295577d483b8fb3230e60dbdfce'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:01.405635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.422923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4ab8-5f7d-4b53-9f04-302fb9224a19 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.431115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.431912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.432425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.432753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] PATH_INFO: `/v3/projects/2a058b52b2ea4935a192da1561d18dd6/users/93672a21fa3941ba8e322869999975b8/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.454273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 924/1837] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:01 2022] DELETE /identity/v3/users/e3159295577d483b8fb3230e60dbdfce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:01.455566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.461706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.462330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a058b52b2ea4935a192da1561d18dd6, user_id=93672a21fa3941ba8e322869999975b8, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.469914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e115cdb-d709-4ed8-ab63-c9a46db51c8d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.497182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 914/1838] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:01 2022] PUT /identity/v3/projects/2a058b52b2ea4935a192da1561d18dd6/users/93672a21fa3941ba8e322869999975b8/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:01.508010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.508380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.508765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.549850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] MFA Rules not processed for user `93672a21fa3941ba8e322869999975b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:01.551647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.574195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0e90-f30f-43b2-ae8d-4826e44b2655 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.609201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.651220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 915/1839] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:01.656735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.657387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.657758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.658043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.658700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.661906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.662838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.664790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.679041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 925/1840] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:01.680438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.693402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4e70-87b2-4ee8-9d6e-e12b974ff57f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.698659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.699330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.699613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.699890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] PATH_INFO: `/v3/projects/8b0d187c5e9f459f847fa9af95387d30/users/c572f1f3863f43f480eaf10aaf94f489/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.717813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0d187c5e9f459f847fa9af95387d30, user_id=c572f1f3863f43f480eaf10aaf94f489, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.722835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653b120-68b5-48b6-81fc-e0b8ddea8e28 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.744053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 926/1841] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:01 2022] PUT /identity/v3/projects/8b0d187c5e9f459f847fa9af95387d30/users/c572f1f3863f43f480eaf10aaf94f489/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:01.754873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.755176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.755460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.758803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37aebb-6f1b-4f84-ad7d-4427221c9636 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.822914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] MFA Rules not processed for user `c572f1f3863f43f480eaf10aaf94f489`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:01.825032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:01.868302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.871184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.871470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.871754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.940625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.989277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 927/1842] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:01.994286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.018383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.022025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.044774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 916/1843] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:02.111803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78566d-bdcf-4354-90e5-a8049304282f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.124333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.125305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.125735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.126212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.127150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.129473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.213271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 928/1844] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:02.214584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.236844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf212e-f3b9-4f37-8ea3-75fd0a25e2e7 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.239962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.242741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.385422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a943ca7-d7c2-4b59-a554-698100300cbe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.396724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.397518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.397671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.398278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] PATH_INFO: `/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.400986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1ef3199e47410ba151cd7edcb4fd5d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.403221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.420080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.420965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.421255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.421582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.422436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.425351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.437529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:02.479398 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 917/1845] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:02.496417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.500706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1ef3199e47410ba151cd7edcb4fd5d', 'request_id': 'req-ab45c6ff-ffaf-4069-9574-e1951455b424'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:02.517595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1ef3199e47410ba151cd7edcb4fd5d', 'request_id': 'req-ab45c6ff-ffaf-4069-9574-e1951455b424'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:02.523045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314f1d0-4232-4882-8207-748ce7e297a8 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.543082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1ef3199e47410ba151cd7edcb4fd5d', 'request_id': 'req-ab45c6ff-ffaf-4069-9574-e1951455b424'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:02.567739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:02.573171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] The token cache is being invalidate because project 8a1ef3199e47410ba151cd7edcb4fd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:02.573558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:02.574394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ab45c6ff-ffaf-4069-9574-e1951455b424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:02.580292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 929/1846] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:02 2022] DELETE /identity/v3/projects/8a1ef3199e47410ba151cd7edcb4fd5d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:02.583058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56123-a661-43b0-af0f-7f67a3bcfe70 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.583676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56123-a661-43b0-af0f-7f67a3bcfe70 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.591124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56123-a661-43b0-af0f-7f67a3bcfe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.657283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-33b56123-a661-43b0-af0f-7f67a3bcfe70 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:02.664690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-33b56123-a661-43b0-af0f-7f67a3bcfe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.706124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.706855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.707033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.707615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.708346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.710951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.723785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 918/1847] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:02.728210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.730986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.747110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49d0e-4518-4804-9067-c8a6b11d635c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.752481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.753323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.753668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.754007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] PATH_INFO: `/v3/projects/8040afa84ef24271ae909838381e1853/users/2bdd6ed792ff4e578793ab7011d46fec/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.772823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8040afa84ef24271ae909838381e1853, user_id=2bdd6ed792ff4e578793ab7011d46fec, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.775710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.791229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 919/1848] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:02 2022] PUT /identity/v3/projects/8040afa84ef24271ae909838381e1853/users/2bdd6ed792ff4e578793ab7011d46fec/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:02.795795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.801721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 930/1849] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:02.812526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.813859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.814154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.845814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:02.847026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.890423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfc97c-7a88-4cd2-9300-cc0589ceaa12 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.895674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.896372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.896663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.897225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.897571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.899293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.915598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 920/1850] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:02.916726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.929179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a93d8-f7a3-4bd5-adda-2234423be4d6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.936493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.937772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.938207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.938858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] PATH_INFO: `/v3/projects/8040afa84ef24271ae909838381e1853/users/2bdd6ed792ff4e578793ab7011d46fec/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.943023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8040afa84ef24271ae909838381e1853, user_id=2bdd6ed792ff4e578793ab7011d46fec, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.948049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09341-7540-4336-ab79-a3c11644d9e1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.965751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 931/1851] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:02.978766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.978969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 921/1852] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:02 2022] PUT /identity/v3/projects/8040afa84ef24271ae909838381e1853/users/2bdd6ed792ff4e578793ab7011d46fec/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:02.984230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126ead-6b52-4eca-8e8f-bfaa67b98952 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.987158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.987437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.987741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.023809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] MFA Rules not processed for user `2bdd6ed792ff4e578793ab7011d46fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:03.025465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.067014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.067410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.068921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.069346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.069760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.070606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:03.071078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.073061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.101431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 932/1853] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:03.110907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.129662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f7a9-de7f-4acc-bcd7-5e454304e0bd admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.135617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 922/1854] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:03.146478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.147182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.147528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.147860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.148637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.151121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.151240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.153628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.183790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 933/1855] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/projects => generated 363 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:03.188091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.203520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.217027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021878f-3645-431d-a338-5f8e8be306f2 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.275699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cc83-56de-4408-97ec-17301d2e1fae admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.280300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.280908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.281180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.281451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.282073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.284014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.284938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.286125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.286621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.286906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.308301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 934/1856] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:03.314621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.324849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512306c5-55a9-4b7e-b911-66025d323cc9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.329463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.330051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.330329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.330660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.331258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.333473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.347599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 935/1857] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:03.349436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.356495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.357932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.358769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d88d-d5c2-4e5d-afe3-f0f124a57a9e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.364738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.365315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.365586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.365856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.366474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.368037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.368870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 923/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:03.378457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 936/1859] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:03.382687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.395223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a44ce-8f1a-4e9a-95d8-38647ad81658 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.399747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.400318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.400980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.401259 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.401532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] PATH_INFO: `/v3/projects/613f1aa7657446839cd4528ff21423da/users/a32031976c344253917bd35e76274ac3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.409891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1dc3-9d7b-45e5-ba6e-fbaf912480c3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.415406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.416354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.416770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.417173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] PATH_INFO: `/v3/users/c572f1f3863f43f480eaf10aaf94f489` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.417358 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613f1aa7657446839cd4528ff21423da, user_id=a32031976c344253917bd35e76274ac3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.418807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c572f1f3863f43f480eaf10aaf94f489)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.420007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9d7c9-d79b-4213-8ecb-1e0fb57dd0b5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.422056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.435851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 924/1860] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:03 2022] PUT /identity/v3/projects/613f1aa7657446839cd4528ff21423da/users/a32031976c344253917bd35e76274ac3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:03.451378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.451649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.451924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.476802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.479783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c572f1f3863f43f480eaf10aaf94f489'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:03.480548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c572f1f3863f43f480eaf10aaf94f489'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:03.488277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] MFA Rules not processed for user `a32031976c344253917bd35e76274ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:03.488991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c572f1f3863f43f480eaf10aaf94f489'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:03.489778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c572f1f3863f43f480eaf10aaf94f489'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:03.489992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.497261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d9e24725-851d-4c12-9b8b-222425390404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c572f1f3863f43f480eaf10aaf94f489'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:03.514765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 937/1861] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:03 2022] DELETE /identity/v3/users/c572f1f3863f43f480eaf10aaf94f489 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:03.519206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.519494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.520346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.542094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:03.543365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.561658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 925/1862] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:03.565720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.567494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.594887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.618657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 938/1863] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:03.629775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.631292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:03.653370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf321c-687f-4404-85c6-9e1431159b78 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.658983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.659645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.659925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.660192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.661935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.666070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.692298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 926/1864] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:03.696278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.700886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b01600-a943-48ac-98dc-b68a951f4ed8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.763555 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.764844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.765123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.765505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.769513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea705a0-c1b4-4fd7-af0b-ea3c2bb67be6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.777048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.777412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.777949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.778370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.778784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.779174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.781920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.785042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.790015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 939/1865] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:03.821255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 927/1866] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:03.822997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.848591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689066c-1cb6-4234-8f62-7b6f630ff388 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.855010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.855566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.855833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.858407 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.858822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.861474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.875748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 940/1867] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:03.880982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.896315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978846-410e-458f-942d-9c2e948d5e70 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.909602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.911593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.912204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.912743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] PATH_INFO: `/v3/projects/55e652cef592444ca713adceb0e2461b/users/8811373063b74484ae4a50e10c7503be/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.935494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.949360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e652cef592444ca713adceb0e2461b, user_id=8811373063b74484ae4a50e10c7503be, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.952860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.980549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.986325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 928/1868] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:03 2022] PUT /identity/v3/projects/55e652cef592444ca713adceb0e2461b/users/8811373063b74484ae4a50e10c7503be/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:03.994045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db8aa-39ee-4b5b-b4ad-168702abbe62 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.085426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.086527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.087471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.088006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.089326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.091860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbc9c7-3f60-40d6-9c68-335cb1f94f82 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.120915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 941/1869] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:04.122028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:04.145603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae61-aabf-4c36-ba00-1c8ee11d0044 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.161855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.161855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.162178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.162437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] PATH_INFO: `/v3/projects/55e652cef592444ca713adceb0e2461b/users/8811373063b74484ae4a50e10c7503be/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.164223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e652cef592444ca713adceb0e2461b, user_id=8811373063b74484ae4a50e10c7503be, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.167791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.181116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 929/1870] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:04 2022] PUT /identity/v3/projects/55e652cef592444ca713adceb0e2461b/users/8811373063b74484ae4a50e10c7503be/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:04.193185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.193646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.193820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.208948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:04.210668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.266939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] MFA Rules not processed for user `8811373063b74484ae4a50e10c7503be`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:04.267744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.317762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:04.369366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e9159-cc3f-437a-897a-d57e79d37b35 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.376654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.377478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.377834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.378176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] PATH_INFO: `/v3/projects/8b0d187c5e9f459f847fa9af95387d30` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.380276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0d187c5e9f459f847fa9af95387d30)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.382591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.437124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 942/1871] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:04.450760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:04.453443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.500687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0d187c5e9f459f847fa9af95387d30', 'request_id': 'req-08f28fac-eef4-4edb-805f-b4146e423fc7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.512635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0d187c5e9f459f847fa9af95387d30', 'request_id': 'req-08f28fac-eef4-4edb-805f-b4146e423fc7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.523504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0d187c5e9f459f847fa9af95387d30', 'request_id': 'req-08f28fac-eef4-4edb-805f-b4146e423fc7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.538993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] The token cache is being invalidate because project 8b0d187c5e9f459f847fa9af95387d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:04.539710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.540837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.541765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86219762-ca2e-4a6b-8e8e-a3126b8fcdb7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.543756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 930/1872] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:04 2022] DELETE /identity/v3/projects/8b0d187c5e9f459f847fa9af95387d30 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:04.548351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:04.550631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.603127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.603747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.604052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.604351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.606259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.626867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28fac-eef4-4edb-805f-b4146e423fc7 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.632547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.633178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.633460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.633750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] PATH_INFO: `/v3/users/2bdd6ed792ff4e578793ab7011d46fec` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.634908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bdd6ed792ff4e578793ab7011d46fec)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.636861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.653124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:04.697660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.699108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.705451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bdd6ed792ff4e578793ab7011d46fec'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.709486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 943/1873] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:04.716983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bdd6ed792ff4e578793ab7011d46fec'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.717938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bdd6ed792ff4e578793ab7011d46fec'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.728776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bdd6ed792ff4e578793ab7011d46fec'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.729627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bdd6ed792ff4e578793ab7011d46fec'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:04.759257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 931/1874] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:04 2022] DELETE /identity/v3/users/2bdd6ed792ff4e578793ab7011d46fec => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:04.792658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:04.794099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.806450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6e0d0-0225-41eb-9c7e-0ecd626423aa service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:04.840731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:04.878038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:04.878487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:04.878791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:04.879216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:04.881687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:04.956729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:04.959035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.969107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 944/1875] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:05.089617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.091092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.195784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.197305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.211845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6617-ec14-4039-be0e-e847c119246a service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.212332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f5d2c-5883-464b-9a73-e39d70279c3f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.216726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.217100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.217360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.217638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.217730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.217912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] PATH_INFO: `/v3/users/93672a21fa3941ba8e322869999975b8` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.218192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.218524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] PATH_INFO: `/v3/projects/8040afa84ef24271ae909838381e1853` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.219145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=93672a21fa3941ba8e322869999975b8)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.219597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8040afa84ef24271ae909838381e1853)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.221111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.221611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.276938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93672a21fa3941ba8e322869999975b8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.277603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93672a21fa3941ba8e322869999975b8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.283224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8040afa84ef24271ae909838381e1853', 'request_id': 'req-813eaffc-afdd-41f6-b2a1-cd22a3f97173'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.289701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93672a21fa3941ba8e322869999975b8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.291481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93672a21fa3941ba8e322869999975b8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.292013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8040afa84ef24271ae909838381e1853', 'request_id': 'req-813eaffc-afdd-41f6-b2a1-cd22a3f97173'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.300706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-57b60cf9-54e5-47d2-9f76-683d8077e43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93672a21fa3941ba8e322869999975b8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.303695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8040afa84ef24271ae909838381e1853', 'request_id': 'req-813eaffc-afdd-41f6-b2a1-cd22a3f97173'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.317187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] The token cache is being invalidate because project 8040afa84ef24271ae909838381e1853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:05.317634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.318540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-813eaffc-afdd-41f6-b2a1-cd22a3f97173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:05.320283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 932/1876] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:05 2022] DELETE /identity/v3/projects/8040afa84ef24271ae909838381e1853 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:05.324821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab40306-b2fc-41c0-bc32-00a0cc635b10 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.325111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab40306-b2fc-41c0-bc32-00a0cc635b10 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.325388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab40306-b2fc-41c0-bc32-00a0cc635b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.325629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 945/1877] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:05 2022] DELETE /identity/v3/users/93672a21fa3941ba8e322869999975b8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:05.337084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.337669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.338049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.345950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6ab40306-b2fc-41c0-bc32-00a0cc635b10 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:05.347157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6ab40306-b2fc-41c0-bc32-00a0cc635b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.358002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:05.359173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.360176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:05.416233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:05.417176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 933/1878] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:05.426376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.426559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.426749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.440347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 946/1879] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:05.450824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.451330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:05.452565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.452692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.465432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 934/1880] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:05.466451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.475976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9026861-47d5-4822-ac1d-803398bcd2a7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.482682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.483411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.483865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.484151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.485556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:05.485871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.488682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.519236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f8a9a-3eaf-4713-8e62-cdd9f18a9d8c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.525873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 935/1881] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:05.527548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.528829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.528829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.528829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.529103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.540601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.542155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efafe46-2f33-462d-a237-39b324615ccf admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.545127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.548137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.549129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.549467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.549746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.550392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.553013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.558183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 947/1882] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:05.586012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 936/1883] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:05.591406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.602422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aceaab-1e2f-4ce6-afc6-14f85ae13688 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.686025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.686346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.686775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.687152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.688083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.690243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.719974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 948/1884] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:05.723974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.736871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6f111-b58f-402c-b3e9-0ceff4d76292 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.748229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.748944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.749223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.749502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.750270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.751936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.766891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 937/1885] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:05.772553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.783365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed155b-e1da-4b42-a9a3-3c1cff01dd4d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.789644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.790141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.790578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.790995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.791503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.793280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7630a-7823-4ae9-87d9-3cb19ae36bbf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.809006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 949/1886] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:05.810765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.849278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc953e1-8491-4534-a191-72f8e35c9f79 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:05.855198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:05.856003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.856380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.857010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] PATH_INFO: `/v3/projects/7a82e9a225274268914fa0f098f2ff66/users/62a0edbac99d4c3796852fb2fe0596e3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.875338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a82e9a225274268914fa0f098f2ff66, user_id=62a0edbac99d4c3796852fb2fe0596e3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:05.878254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:05.904988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 938/1887] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:05 2022] PUT /identity/v3/projects/7a82e9a225274268914fa0f098f2ff66/users/62a0edbac99d4c3796852fb2fe0596e3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:05.913759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.913759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.913759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.938662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] MFA Rules not processed for user `62a0edbac99d4c3796852fb2fe0596e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:05.939936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.947385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:05.949740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.021373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 950/1888] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:06.025859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.034432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.052671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e5bd4-610a-4599-970d-3fe9d0ec770a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.057419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.058084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.058364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.058636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] PATH_INFO: `/v3/projects/2a058b52b2ea4935a192da1561d18dd6` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.059786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a058b52b2ea4935a192da1561d18dd6)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.061771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.105772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28c31a-d756-4a6e-bbb0-112c1933f4fb None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.115491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.119917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a058b52b2ea4935a192da1561d18dd6', 'request_id': 'req-a7ce78fa-8e0e-4f32-afaf-3b131b753534'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:06.127194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a058b52b2ea4935a192da1561d18dd6', 'request_id': 'req-a7ce78fa-8e0e-4f32-afaf-3b131b753534'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:06.138397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a058b52b2ea4935a192da1561d18dd6', 'request_id': 'req-a7ce78fa-8e0e-4f32-afaf-3b131b753534'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:06.153177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] The token cache is being invalidate because project 2a058b52b2ea4935a192da1561d18dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:06.154193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:06.154888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a7ce78fa-8e0e-4f32-afaf-3b131b753534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:06.162787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 939/1889] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:05 2022] DELETE /identity/v3/projects/2a058b52b2ea4935a192da1561d18dd6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:06.164365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa494-4bba-4f9e-92eb-d9058f139070 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.164656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa494-4bba-4f9e-92eb-d9058f139070 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.164928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa494-4bba-4f9e-92eb-d9058f139070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.189902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fcdaa494-4bba-4f9e-92eb-d9058f139070 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:06.191104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fcdaa494-4bba-4f9e-92eb-d9058f139070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.205522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.206279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.206685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.207101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.208227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.210300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.258027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 951/1890] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:06.260344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.262819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.288956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 940/1891] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:06.297568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.297880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.298291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.323287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:06.324596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.356149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca8f6-1933-4c9f-8bcf-de3b8ba45dc2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.362223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.364681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.365155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.365559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.366298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.368790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.400437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 952/1892] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:06.406895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 941/1893] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:06.407079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.409618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.417120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315deff9-06eb-4a2b-9a14-b941160c36f6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.422807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.423424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.423703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.423977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.424832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.425116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26be11c-6914-4714-96d6-f38abf992cc5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.427949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.434501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.435919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.436615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.436898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.437858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:06.438161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.440450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.444353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 953/1894] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:06.447163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.459768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 942/1895] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:06.461936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872efe5-ea3e-48ea-bf91-a9cc3c5df4e1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.463950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.474849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.475865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.477704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cefb41-8f57-412a-9ea3-f88fe691435a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.477950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.478406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.479281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.481852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.484034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.485145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.485478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.486113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.486437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.489612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.502663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 954/1896] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:06.504658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.512500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.517871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef928-4c90-4bcf-8388-7daf251c3aa4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.526999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.527936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.528655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.528945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] PATH_INFO: `/v3/projects/2ed0c67f7753458ba2b3c3616c9dacf9/users/516844271cdf4e32935acba3d72f2f06/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.539413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 943/1897] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/projects => generated 397 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:06.544217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.550640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed0c67f7753458ba2b3c3616c9dacf9, user_id=516844271cdf4e32935acba3d72f2f06, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.554044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902b7f-237b-4364-a5f9-d787168bc12b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.581623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 955/1898] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:06 2022] PUT /identity/v3/projects/2ed0c67f7753458ba2b3c3616c9dacf9/users/516844271cdf4e32935acba3d72f2f06/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:06.584656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.584930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.585184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.601009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.623209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] MFA Rules not processed for user `516844271cdf4e32935acba3d72f2f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:06.624651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.632324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057684db-bcba-41c3-8312-6ebd07e81ee1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.716438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 956/1899] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:06.728058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.743034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.743671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.744549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.744836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.745643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.749472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.784911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 944/1900] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/users => generated 299 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:06.789093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.810989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92625e1-d72f-4815-8a34-63a8d1086029 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.822812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.823958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.824476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.825028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.826088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.829823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.835392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bc54c-9ada-452c-be38-d704be0272e0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.841227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.842028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.842310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.842701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.843307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.847342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 945/1901] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:06.847482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.852398 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.869714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6645f16-84ac-47d7-8558-23fb4ae94f6d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.877396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.878445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.879690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.880663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.881455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.883477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.888552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 957/1902] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:06.889740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.899857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 946/1903] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:06.903752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.914054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec73be-9186-4487-ad57-bb915b36742f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.962936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.969097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763a1b-90e5-420a-be2b-f20e5f877442 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.975948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.976606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.976896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.977149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.977774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.979592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.979714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.980367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.980720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.981067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] PATH_INFO: `/v3/projects/69e743cc488a4678af30dba48abc536c/users/f881a8c283de4d13afd322a6ab7d3a6b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.999812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e743cc488a4678af30dba48abc536c, user_id=f881a8c283de4d13afd322a6ab7d3a6b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.002861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9056df-724f-46de-9e16-8ac506e6a21d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.009458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 958/1904] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:07.010447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.022495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 947/1905] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:06 2022] PUT /identity/v3/projects/69e743cc488a4678af30dba48abc536c/users/f881a8c283de4d13afd322a6ab7d3a6b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:07.022853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a08c-5084-4fe1-a70d-ea9607172592 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.032661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.032950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.033227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.062711 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] MFA Rules not processed for user `f881a8c283de4d13afd322a6ab7d3a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:07.063995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:07.093299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.093906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.094309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.094591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.095041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.095498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.098036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.120822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 959/1906] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:07.122778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.136666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880dab7-1250-47ef-8d46-c8a71779bcdc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.144172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.144930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.145207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.145575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] PATH_INFO: `/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1/users/81c7c316036c489b8700ff3367dc0835/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.162595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 948/1907] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:07.171994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dddb9fda3c8442ca0ba8bd61d567cf1, user_id=81c7c316036c489b8700ff3367dc0835, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.175514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.176431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.178190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:07.198578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 960/1908] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:07 2022] PUT /identity/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1/users/81c7c316036c489b8700ff3367dc0835/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:07.202328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.255822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f42b-183c-4e42-9268-4d37eab73355 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.277313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2cfe-f6ff-4833-b94e-08a7e12f3675 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.282437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.283578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.283578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.283754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.284276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.286550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.301830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 961/1909] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:07.302936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.314873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608b6ae-1ff8-401f-a03f-649355828cf1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.319993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.320700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.321112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.321505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] PATH_INFO: `/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1/users/81c7c316036c489b8700ff3367dc0835/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.323442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dddb9fda3c8442ca0ba8bd61d567cf1, user_id=81c7c316036c489b8700ff3367dc0835, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.326525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460909e5-0ccd-4587-9cd1-a4fef72605bd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.336531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.337702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.346540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 962/1910] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:07 2022] PUT /identity/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1/users/81c7c316036c489b8700ff3367dc0835/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:07.346692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.346692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.349825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.350120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.350395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.391246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] MFA Rules not processed for user `81c7c316036c489b8700ff3367dc0835`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:07.393448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:07.421356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.423056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.438137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.446419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 949/1911] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:07.512331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 963/1912] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:07.525961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.536404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173f695-715b-49bb-b25a-32ad8bc8f9bc service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.587994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.617535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.618243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.618589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.619090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.643933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.645721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.659194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 950/1913] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:08.534321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.535814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:08.547443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86667391-ed89-4fdc-901f-ec5c38cc1ccd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.552089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.552808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.553050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.553327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.557280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:08.625914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.627976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bf579-f278-449c-8077-3e4044b6717f service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.639494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 964/1914] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:17.653359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=10 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.635170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=44 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.355783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.357294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:20.427851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12309b-6a10-4bd4-899c-42c97a4dd2de service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.432378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.433046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.433350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.433615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] PATH_INFO: `/v3/users/9a8467a69b7c45348f7cf66e10f24c89` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.434730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8467a69b7c45348f7cf66e10f24c89)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.436682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.492104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8467a69b7c45348f7cf66e10f24c89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:20.503284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8467a69b7c45348f7cf66e10f24c89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:20.504006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8467a69b7c45348f7cf66e10f24c89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:20.512582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8467a69b7c45348f7cf66e10f24c89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:20.513397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8467a69b7c45348f7cf66e10f24c89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:20.534113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 951/1915] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:20 2022] DELETE /identity/v3/users/9a8467a69b7c45348f7cf66e10f24c89 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:20.538089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.538558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.538887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.562031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:20.563237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:20.632303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 965/1916] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:20.643231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.647761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:20.709595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4454-6eee-45e8-b94b-752e82ce55cf admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.714108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.714730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.715003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.715273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.726116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.727729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.738356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 952/1917] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:21.026896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.028369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.040500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814efa8-b830-4146-9cdb-8175d35f68f5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.046967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.047632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.047944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.048130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] PATH_INFO: `/v3/projects/cf1abe56cb75460798bc84ff453ed09d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.049374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1abe56cb75460798bc84ff453ed09d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.051262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.102950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1abe56cb75460798bc84ff453ed09d', 'request_id': 'req-b8faa437-7c61-432d-a72f-af308a6fbd10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.111254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1abe56cb75460798bc84ff453ed09d', 'request_id': 'req-b8faa437-7c61-432d-a72f-af308a6fbd10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.119372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1abe56cb75460798bc84ff453ed09d', 'request_id': 'req-b8faa437-7c61-432d-a72f-af308a6fbd10'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.131919 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] The token cache is being invalidate because project cf1abe56cb75460798bc84ff453ed09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:21.132172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.132979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.134704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 966/1918] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:21 2022] DELETE /identity/v3/projects/cf1abe56cb75460798bc84ff453ed09d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:21.138920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.149573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2a03-ef3c-431f-befb-0969b7d5e97e service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.168102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.213433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.214258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.214801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.215257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] PATH_INFO: `/v3/users/f924e3d498e0455c934e95deaa8648b0` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.216555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f924e3d498e0455c934e95deaa8648b0)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.218813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.291628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f924e3d498e0455c934e95deaa8648b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.304569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.304569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.304877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f924e3d498e0455c934e95deaa8648b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.304877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f924e3d498e0455c934e95deaa8648b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.323892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faa437-7c61-432d-a72f-af308a6fbd10 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.336607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f924e3d498e0455c934e95deaa8648b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.338648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f924e3d498e0455c934e95deaa8648b0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.341194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.341971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.342579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.343113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] PATH_INFO: `/v3/users/f881a8c283de4d13afd322a6ab7d3a6b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.346318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f881a8c283de4d13afd322a6ab7d3a6b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.350909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.387011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 953/1919] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:21 2022] DELETE /identity/v3/users/f924e3d498e0455c934e95deaa8648b0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:21.434022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f881a8c283de4d13afd322a6ab7d3a6b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.434853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f881a8c283de4d13afd322a6ab7d3a6b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.448346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f881a8c283de4d13afd322a6ab7d3a6b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.450113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f881a8c283de4d13afd322a6ab7d3a6b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.472680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6f003665-fcf4-4b47-983a-a44664619a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f881a8c283de4d13afd322a6ab7d3a6b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:21.511499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.532766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.534501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.538232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 967/1920] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:21 2022] DELETE /identity/v3/users/f881a8c283de4d13afd322a6ab7d3a6b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:21.540965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.541434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.541601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.575487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:21.576106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.669123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1bda4-43ff-4112-8040-5b9107053475 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.715276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 968/1921] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:21.731296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.733556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.760728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.761518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.762840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.763190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.770410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:21.817407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.845974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27403bc3-ff60-4bff-ba33-00ef9ea492e1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.855211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.856184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.856690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.857172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.873023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.874913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.888859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 969/1922] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:21.890196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.891833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01cd43-7f84-4acf-9725-6012778dad65 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.904292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 954/1923] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:21.982471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.985365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.012552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6be6e-6bce-4804-b8a6-707a9a71ab0a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.019738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.021692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.022270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.022793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] PATH_INFO: `/v3/projects/e50ecbdcbc17481894af5685223413ad` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.024206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50ecbdcbc17481894af5685223413ad)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.026247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.108982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50ecbdcbc17481894af5685223413ad', 'request_id': 'req-c0979c49-dc70-4c83-85c0-ae0615628dbb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.119698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50ecbdcbc17481894af5685223413ad', 'request_id': 'req-c0979c49-dc70-4c83-85c0-ae0615628dbb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.127341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50ecbdcbc17481894af5685223413ad', 'request_id': 'req-c0979c49-dc70-4c83-85c0-ae0615628dbb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.141171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] The token cache is being invalidate because project e50ecbdcbc17481894af5685223413ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:22.141496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.142455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.144593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 970/1924] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:21 2022] DELETE /identity/v3/projects/e50ecbdcbc17481894af5685223413ad => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:22.149900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196879-ce6d-4fc9-b686-86e40871e833 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.150865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196879-ce6d-4fc9-b686-86e40871e833 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.150997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196879-ce6d-4fc9-b686-86e40871e833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.174992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-56196879-ce6d-4fc9-b686-86e40871e833 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:22.176220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-56196879-ce6d-4fc9-b686-86e40871e833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.243459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.246129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.267388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 955/1925] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:22.277315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.277680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.278077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.291106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.309849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:22.311107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.325297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 956/1926] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:22.329824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.340329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fa13a-cadd-415d-b42b-f97db92a420e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.343321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979c49-dc70-4c83-85c0-ae0615628dbb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.345051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.345658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.345928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.346193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.346851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:22.347147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.348754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.348907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.350143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.350572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.350923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] PATH_INFO: `/v3/projects/69e743cc488a4678af30dba48abc536c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.352062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e743cc488a4678af30dba48abc536c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.353995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.361494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 957/1927] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:22.365880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.377600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970999c-d1d9-48ac-abbc-ee48f2c354fe admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.383284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.384160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.384587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.384936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.385733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.388883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.418469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e743cc488a4678af30dba48abc536c', 'request_id': 'req-fa2c916c-1528-421c-93a0-4337e0b97ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.425881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 958/1928] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:22.430297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.431451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e743cc488a4678af30dba48abc536c', 'request_id': 'req-fa2c916c-1528-421c-93a0-4337e0b97ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.448050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e743cc488a4678af30dba48abc536c', 'request_id': 'req-fa2c916c-1528-421c-93a0-4337e0b97ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.460977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] The token cache is being invalidate because project 69e743cc488a4678af30dba48abc536c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:22.461270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.462320 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa2c916c-1528-421c-93a0-4337e0b97ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:22.463921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 971/1929] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:22 2022] DELETE /identity/v3/projects/69e743cc488a4678af30dba48abc536c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:22.480725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce85e-2829-4447-a98e-ac37c5c0e890 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.480725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce85e-2829-4447-a98e-ac37c5c0e890 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.480725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ce85e-2829-4447-a98e-ac37c5c0e890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.495450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-698ce85e-2829-4447-a98e-ac37c5c0e890 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:22.496685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-698ce85e-2829-4447-a98e-ac37c5c0e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.520793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec2d0-f8b5-4e92-a589-1fb5a7849142 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.527393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.528117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.528440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.528744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.529352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.531588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.550673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.558108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 959/1930] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:22.562101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.564826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.576643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 972/1931] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:22.578029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.578385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.578650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28b288-5c1f-46b0-8644-89e7ddd8153e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.578759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.584211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.584914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.585195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.585467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.586105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.587670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.598059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 960/1932] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:22.600597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:22.602016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.603035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.611672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 973/1933] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:22.613462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8444a0c-e93b-4755-987f-9ee95671a565 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.617305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.619243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.619839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.620117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.620513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.621032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.622605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.627127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0c987-0e4a-4f48-8e72-ab6cf492b31d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.634640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.635339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.635818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.636178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.637004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:22.639523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.639658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 961/1934] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:22.643393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.651684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.654625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65cbe7-7331-42f7-b34b-c6ad69611f41 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.659388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.660135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.660600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.660988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] PATH_INFO: `/v3/projects/ef47aeb4fad64304b0444d38c95a5bfb/users/de7af0f5a58d4bf68c8e51d5cc707f29/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.668370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 974/1935] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:22.669950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.676940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef47aeb4fad64304b0444d38c95a5bfb, user_id=de7af0f5a58d4bf68c8e51d5cc707f29, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.680226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b851a-c5e8-41db-bc16-dbc22f394579 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.680816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915a228-06e1-45c2-a011-5a51c0937eda admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.685739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.686423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.686761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.687095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.687790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.689791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.696007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 962/1936] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:22 2022] PUT /identity/v3/projects/ef47aeb4fad64304b0444d38c95a5bfb/users/de7af0f5a58d4bf68c8e51d5cc707f29/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:22.707085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f10e3-0ef8-4918-af67-20762559515e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.707380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f10e3-0ef8-4918-af67-20762559515e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.707655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f10e3-0ef8-4918-af67-20762559515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.724234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 975/1937] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:22.725522 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.740753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-751f10e3-0ef8-4918-af67-20762559515e None None] MFA Rules not processed for user `de7af0f5a58d4bf68c8e51d5cc707f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:22.741441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.741967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-751f10e3-0ef8-4918-af67-20762559515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:22.793686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe2515-c659-48f9-a0ee-038d1f302a12 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.798949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.799551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.799827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.800097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.800808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.803849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.823975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 963/1938] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:22.825329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 976/1939] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:22.828616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f10e3-0ef8-4918-af67-20762559515e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.829640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.839947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f10e3-0ef8-4918-af67-20762559515e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.844460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080821-3dcb-458c-84fd-19a60eb9bcd1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.847158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.847987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.848474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.848745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.849529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.849785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.850374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.850532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.850781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.851414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.851894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.853031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.865888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 977/1940] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:22.869492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.879658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ebde-1c00-4fb9-8c0b-fdc9c1331ab0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.888768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.889376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.889653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.889922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.890551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.892128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.902896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 964/1941] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:22.904310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 978/1942] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:22.907096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.908268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.922085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411b564-9420-48e0-9c49-51bb9d9d7071 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.995185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7264-45b0-456d-824c-8a20d002b059 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.000243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.000875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.001162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.001402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.001577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.002037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.002257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.002716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.002968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] PATH_INFO: `/v3/projects/e1b5f9daf8d44a6987bbff8939ed7c92/users/a0794e19f43a413db886750b0517d3d3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.004170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.026485 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b5f9daf8d44a6987bbff8939ed7c92, user_id=a0794e19f43a413db886750b0517d3d3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.032853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a736fa-e563-4e32-951d-73351ce5ee77 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.036954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 965/1943] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:22 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:23.041311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.052975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf489198-573d-4074-94a7-f838b6d13eb4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.062425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 979/1944] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:22 2022] PUT /identity/v3/projects/e1b5f9daf8d44a6987bbff8939ed7c92/users/a0794e19f43a413db886750b0517d3d3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:23.063950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.064424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.064738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.070745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.087185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:23.088690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.144428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.145042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.145321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.145603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.146294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.148073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.159076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 966/1945] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:23.160195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.163679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.173622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 980/1946] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:23.174840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.180597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.181650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8d556-37dd-4bfd-94b1-02e020729836 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.187073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba53d2-ee12-4edf-a7ff-6ab76da7e4ea None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.188346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.189211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.189838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.190395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] PATH_INFO: `/v3/users/a32031976c344253917bd35e76274ac3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.191571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32031976c344253917bd35e76274ac3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.194492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.195231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.196463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.196812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.197513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] PATH_INFO: `/v3/projects/b338ec63500e45a68fcf31ab48c39cf7/users/8782078668e64ed19964c7559ebfda94/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.213412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b338ec63500e45a68fcf31ab48c39cf7, user_id=8782078668e64ed19964c7559ebfda94, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.216126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.231610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 981/1947] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:23 2022] PUT /identity/v3/projects/b338ec63500e45a68fcf31ab48c39cf7/users/8782078668e64ed19964c7559ebfda94/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:23.232597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.234562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.255325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32031976c344253917bd35e76274ac3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:23.265555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32031976c344253917bd35e76274ac3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:23.266385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32031976c344253917bd35e76274ac3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:23.275830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32031976c344253917bd35e76274ac3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:23.276596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32031976c344253917bd35e76274ac3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:23.295046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 967/1948] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:23 2022] DELETE /identity/v3/users/a32031976c344253917bd35e76274ac3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:23.296202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.297720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.298049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16827029-fcd0-4d2e-987f-df0fc8ccb000 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.347077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.347689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.347966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.348233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.350515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.362197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.362673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddc922-7beb-442e-8442-3726c41904f2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.363623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61570d9f-998e-48f7-8870-fe1851e6979e service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.368021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.368655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.368932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.369203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.369838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.371415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.373987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 982/1949] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:23.376308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.376308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.376308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.382609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 968/1950] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:23.387211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.395784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:23.397406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.398692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5f8b-28d8-463d-9f44-744697f47a24 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.404671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.405613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.405895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.406182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 983/1951] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:23.408885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] PATH_INFO: `/v3/projects/b338ec63500e45a68fcf31ab48c39cf7/users/8782078668e64ed19964c7559ebfda94/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.411157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b338ec63500e45a68fcf31ab48c39cf7, user_id=8782078668e64ed19964c7559ebfda94, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.415007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2e755-673c-4a11-9332-0aada3566fa1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.416581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.418019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.427972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 969/1952] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:23 2022] PUT /identity/v3/projects/b338ec63500e45a68fcf31ab48c39cf7/users/8782078668e64ed19964c7559ebfda94/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:23.430705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a98223-a960-472e-8315-0b9c336022f8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.439307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.440324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.440443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.449211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.483041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] MFA Rules not processed for user `8782078668e64ed19964c7559ebfda94`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:23.484774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.527875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.528864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.529251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.529628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.583321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 970/1953] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:23.606052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.607559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:23.619829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.651769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.653252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.666482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 984/1954] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:23.676643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36b6c-798d-4a40-96ef-d7d774419fd8 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.680865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.681488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.681937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.682164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.692189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.694354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.707842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 971/1955] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:24.017538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.019356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:24.062911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3428-e793-484e-8e66-ec0ca8e33c09 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.069890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.075302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.075734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.076168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] PATH_INFO: `/v3/projects/613f1aa7657446839cd4528ff21423da` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.087775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] RBAC: Authorizing `identity:delete_project(project_id=613f1aa7657446839cd4528ff21423da)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.089754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.151606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613f1aa7657446839cd4528ff21423da', 'request_id': 'req-5ae433ff-52cc-4ce2-8055-52567be0797b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.163723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613f1aa7657446839cd4528ff21423da', 'request_id': 'req-5ae433ff-52cc-4ce2-8055-52567be0797b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.176667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613f1aa7657446839cd4528ff21423da', 'request_id': 'req-5ae433ff-52cc-4ce2-8055-52567be0797b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.197117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] The token cache is being invalidate because project 613f1aa7657446839cd4528ff21423da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:24.197410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.198120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.202785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 985/1956] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:24 2022] DELETE /identity/v3/projects/613f1aa7657446839cd4528ff21423da => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:24.205305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.219494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b08b-efe4-4a02-ad5a-9c7992524176 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.359031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.360363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.360607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.360923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] PATH_INFO: `/v3/users/8811373063b74484ae4a50e10c7503be` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.362365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8811373063b74484ae4a50e10c7503be)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.365447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.439279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:24.466982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8811373063b74484ae4a50e10c7503be'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.485069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8811373063b74484ae4a50e10c7503be'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.487062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8811373063b74484ae4a50e10c7503be'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.500809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8811373063b74484ae4a50e10c7503be'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.501980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c51d8ddf-1fb1-4f87-81b2-d47b26ce7b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8811373063b74484ae4a50e10c7503be'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.535806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 972/1957] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:24 2022] DELETE /identity/v3/users/8811373063b74484ae4a50e10c7503be => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:24.750861 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.753015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:24.775150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae433ff-52cc-4ce2-8055-52567be0797b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.890805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.892689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.893028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.893717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] PATH_INFO: `/v3/users/62a0edbac99d4c3796852fb2fe0596e3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.895194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a0edbac99d4c3796852fb2fe0596e3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.898734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.975807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a0edbac99d4c3796852fb2fe0596e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.976610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a0edbac99d4c3796852fb2fe0596e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.994507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a0edbac99d4c3796852fb2fe0596e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:24.996644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a0edbac99d4c3796852fb2fe0596e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.006114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a0edbac99d4c3796852fb2fe0596e3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.015105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:25.070955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 986/1958] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:24 2022] DELETE /identity/v3/users/62a0edbac99d4c3796852fb2fe0596e3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:25.073123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.073825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.074846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.125658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:25.127558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.230131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 973/1959] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:25.241640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.242855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.257053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.276996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.304478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732d3b1-257c-4bc5-b093-2ca0c3bbd90c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.310432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.311075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.311350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.311615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] PATH_INFO: `/v3/projects/55e652cef592444ca713adceb0e2461b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.312894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e652cef592444ca713adceb0e2461b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.314770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.380782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b95f-b15a-4aa8-b6e3-9987086c2fd5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.405132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e652cef592444ca713adceb0e2461b', 'request_id': 'req-bd178af2-5964-41e6-881b-8372fa5792f7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.414000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e652cef592444ca713adceb0e2461b', 'request_id': 'req-bd178af2-5964-41e6-881b-8372fa5792f7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.437270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e652cef592444ca713adceb0e2461b', 'request_id': 'req-bd178af2-5964-41e6-881b-8372fa5792f7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.456164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] The token cache is being invalidate because project 55e652cef592444ca713adceb0e2461b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:25.456774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.457549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bd178af2-5964-41e6-881b-8372fa5792f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:25.460365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 974/1960] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:25 2022] DELETE /identity/v3/projects/55e652cef592444ca713adceb0e2461b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:25.465342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1b0e-9fec-4fac-93b0-dc4700e6cdfc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.465732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1b0e-9fec-4fac-93b0-dc4700e6cdfc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.467690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.468527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.468663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f1b0e-9fec-4fac-93b0-dc4700e6cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.469067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.469483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.471451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.500096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-932f1b0e-9fec-4fac-93b0-dc4700e6cdfc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:25.501322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-932f1b0e-9fec-4fac-93b0-dc4700e6cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.592037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:25.667231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 975/1961] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:25.667892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.668923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.669198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.669476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.670793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.701827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 987/1962] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:25.719454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:25.720755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:25.735294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 976/1963] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:25.741415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.763903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cc74d-df1c-4f62-a708-d03bb2ae8bf6 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.782105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.784383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.784720 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.785398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.788102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:25.789578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.793537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.847987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 988/1964] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:25.852858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.866839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e88cbe-bdd3-42ee-a2f0-9e4a845619d4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.877824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.878486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.878779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.879073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.879746 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.882699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.946187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 977/1965] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:25.950496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.970387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ececff5-ee3f-4870-814e-17cf6cd583ee admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.044716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:26.075553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.077079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.077504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.077974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.079278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.082301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.119543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 989/1966] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:26.123462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.136463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1227f58-a0ff-48fa-8532-1c121f3bd604 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.141493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.144224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.157301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.158306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.158767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53741a-aad4-4b44-8f1a-f7bbd04744d3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.159123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.159605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.160649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.164584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.166175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.167348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.167348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.167545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] PATH_INFO: `/v3/users/de7af0f5a58d4bf68c8e51d5cc707f29` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.168931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7af0f5a58d4bf68c8e51d5cc707f29)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.171565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.194552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 978/1967] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:26.196806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.212673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25942-42c4-45bc-8475-14ee551e5acb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.218333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.219142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.219596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.220038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.220941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.223718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.243979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 979/1968] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:26.245446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.248794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.267818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377958-2e1e-47a7-a6ec-5ea66b1a5de9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.275405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7af0f5a58d4bf68c8e51d5cc707f29'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.276733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7af0f5a58d4bf68c8e51d5cc707f29'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.285638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.286192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.287166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.287745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] PATH_INFO: `/v3/projects/7a82e9a225274268914fa0f098f2ff66` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.290114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a82e9a225274268914fa0f098f2ff66)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.291902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7af0f5a58d4bf68c8e51d5cc707f29'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.293270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7af0f5a58d4bf68c8e51d5cc707f29'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.297492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.310641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7af0f5a58d4bf68c8e51d5cc707f29'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.336232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 990/1969] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:26 2022] DELETE /identity/v3/users/de7af0f5a58d4bf68c8e51d5cc707f29 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:26.337724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.341169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.390507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a82e9a225274268914fa0f098f2ff66', 'request_id': 'req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.400517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a82e9a225274268914fa0f098f2ff66', 'request_id': 'req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.414401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a82e9a225274268914fa0f098f2ff66', 'request_id': 'req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.428600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] The token cache is being invalidate because project 7a82e9a225274268914fa0f098f2ff66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:26.428885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.429616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-373eea7d-11d8-4763-b5e5-9dbab23dc9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.434367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 980/1970] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:26 2022] DELETE /identity/v3/projects/7a82e9a225274268914fa0f098f2ff66 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:26.434367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.434367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.434367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.444775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4e77f-643c-4bf2-b694-9e596a79a1e3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.459431 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:26.460699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.506718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:26.534130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.534992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.535333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.535674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] PATH_INFO: `/v3/projects/b792b66b7d974b65b05c68eeb28c76b7/users/b2140cb87fc54cceb72096610a355b5d/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.542759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 981/1971] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:26.544889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.546939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.561194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93cd13-7124-45da-a358-e6f9f5d2fb04 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.567601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.568584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.569005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.569278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] PATH_INFO: `/v3/users/516844271cdf4e32935acba3d72f2f06` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.570524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=516844271cdf4e32935acba3d72f2f06)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.571697 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b792b66b7d974b65b05c68eeb28c76b7, user_id=b2140cb87fc54cceb72096610a355b5d, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.572410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.574593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:26.578116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.611182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 991/1972] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:26 2022] PUT /identity/v3/projects/b792b66b7d974b65b05c68eeb28c76b7/users/b2140cb87fc54cceb72096610a355b5d/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:26.612201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.614010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.649114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516844271cdf4e32935acba3d72f2f06'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.663329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516844271cdf4e32935acba3d72f2f06'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.664981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516844271cdf4e32935acba3d72f2f06'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.678649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516844271cdf4e32935acba3d72f2f06'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.679658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b08ab05-f86e-4612-a8f0-747b0683b9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516844271cdf4e32935acba3d72f2f06'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:26.706145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 982/1973] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:26 2022] DELETE /identity/v3/users/516844271cdf4e32935acba3d72f2f06 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:26.707703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.708821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.709138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.713305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f7ff-0d4d-4063-8f11-e9e89b0479e7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.758126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] MFA Rules not processed for user `b2140cb87fc54cceb72096610a355b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:26.759911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.821252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.821882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.822160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.822466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.824415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.874925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 983/1974] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:26.880745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.882928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:27.003777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f7faa-32b5-40da-94bb-183a7fd3eff6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.010381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.011497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.011877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.012306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.013380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.015586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.029331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.030786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.033287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:27.060115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 992/1975] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:27.060706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 984/1976] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:26 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:27.061322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.087190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d3ebd-ee15-4e23-80b6-f6a76e9e5df8 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.142260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:27.200363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.201076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.201367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.201626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.202268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.204236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.251639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 993/1977] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:27 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:27.256468 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.273683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c3b4-b263-4373-85ef-a460bf1b4b67 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.282645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.288857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.289429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.289950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.290853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.292949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.324362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 985/1978] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:27.327948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.345191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87bae3-864d-413d-b3c6-743f15266b62 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.352448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.353565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.353966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.354340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] PATH_INFO: `/v3/projects/08c3961ab4044c35b585b1b52c4e80f9/users/16d2145b9063476982120121c99884fd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.401075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c3961ab4044c35b585b1b52c4e80f9, user_id=16d2145b9063476982120121c99884fd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.405607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.423670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 994/1979] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:27 2022] PUT /identity/v3/projects/08c3961ab4044c35b585b1b52c4e80f9/users/16d2145b9063476982120121c99884fd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:27.428142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.441336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91320c6-8596-44e5-a164-a7a2774cde5b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.556450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.557235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.557235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.557544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.558173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.559834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c325d-7879-47ae-9095-d220355e5c84 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.573186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 986/1980] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:27.577326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.593100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9d943-1ef8-4436-8831-d9742d9d7c21 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.603412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.604305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.604679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.605049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] PATH_INFO: `/v3/projects/08c3961ab4044c35b585b1b52c4e80f9/users/16d2145b9063476982120121c99884fd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.607258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c3961ab4044c35b585b1b52c4e80f9, user_id=16d2145b9063476982120121c99884fd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.612076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.644139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 995/1981] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:27 2022] PUT /identity/v3/projects/08c3961ab4044c35b585b1b52c4e80f9/users/16d2145b9063476982120121c99884fd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:27.646164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.646697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.647176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.668274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.670412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:27.707818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] MFA Rules not processed for user `16d2145b9063476982120121c99884fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:27.709173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:27.790228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7454-b178-42a6-99a7-e53a7cb208e7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.795655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.796319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.796601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.796867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] PATH_INFO: `/v3/projects/ef47aeb4fad64304b0444d38c95a5bfb` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.798112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef47aeb4fad64304b0444d38c95a5bfb)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.800721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.874789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 987/1982] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:27.888170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.889742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:27.897629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef47aeb4fad64304b0444d38c95a5bfb', 'request_id': 'req-f3931976-f64c-4494-b036-0e47d15ebb42'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:27.910946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef47aeb4fad64304b0444d38c95a5bfb', 'request_id': 'req-f3931976-f64c-4494-b036-0e47d15ebb42'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:27.925492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef47aeb4fad64304b0444d38c95a5bfb', 'request_id': 'req-f3931976-f64c-4494-b036-0e47d15ebb42'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:27.946916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] The token cache is being invalidate because project ef47aeb4fad64304b0444d38c95a5bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:27.946916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:27.947765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:27.949381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 996/1983] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:27 2022] DELETE /identity/v3/projects/ef47aeb4fad64304b0444d38c95a5bfb => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:27.953460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.955126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:27.981654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.004782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ba18-d39e-4682-9168-ec07fdadbbf6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.023511 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.066332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931976-f64c-4494-b036-0e47d15ebb42 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.071305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.071761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.071939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.072195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.072444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.072563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.072722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.072999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] PATH_INFO: `/v3/users/8782078668e64ed19964c7559ebfda94` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.074143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8782078668e64ed19964c7559ebfda94)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.074256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.076071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.234521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8782078668e64ed19964c7559ebfda94'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.237915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8782078668e64ed19964c7559ebfda94'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.257524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8782078668e64ed19964c7559ebfda94'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.258260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8782078668e64ed19964c7559ebfda94'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.269393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.270925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.281393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8782078668e64ed19964c7559ebfda94'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.311444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 988/1984] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:28.314304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.317008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.327032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 997/1985] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:27 2022] DELETE /identity/v3/users/8782078668e64ed19964c7559ebfda94 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:28.338802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f123b-a2e6-4c61-9ef0-189dadcc3483 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.408458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.409930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.498535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.499863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.500141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.500429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] PATH_INFO: `/v3/projects/2ed0c67f7753458ba2b3c3616c9dacf9` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.502312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed0c67f7753458ba2b3c3616c9dacf9)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.504220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.526128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05472ef-3f17-4829-ab48-c222621f4cfc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.534676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.535622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.535830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.536739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.542386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.613042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed0c67f7753458ba2b3c3616c9dacf9', 'request_id': 'req-5c9a72f4-d686-48e9-ab62-da4d378c8124'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.629183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed0c67f7753458ba2b3c3616c9dacf9', 'request_id': 'req-5c9a72f4-d686-48e9-ab62-da4d378c8124'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.645226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed0c67f7753458ba2b3c3616c9dacf9', 'request_id': 'req-5c9a72f4-d686-48e9-ab62-da4d378c8124'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.665708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.671445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] The token cache is being invalidate because project 2ed0c67f7753458ba2b3c3616c9dacf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:28.672334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.675174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.679021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 989/1986] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:28 2022] DELETE /identity/v3/projects/2ed0c67f7753458ba2b3c3616c9dacf9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:28.684917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.689256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.693083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.713783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.715639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.730559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 998/1987] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:28.801338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a72f4-d686-48e9-ab62-da4d378c8124 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.809286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.810155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.810678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.811475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] PATH_INFO: `/v3/users/81c7c316036c489b8700ff3367dc0835` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.817521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c7c316036c489b8700ff3367dc0835)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.819587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.903474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c7c316036c489b8700ff3367dc0835'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.914759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c7c316036c489b8700ff3367dc0835'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.915436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c7c316036c489b8700ff3367dc0835'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.934489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c7c316036c489b8700ff3367dc0835'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.935445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c46ae6aa-6bc7-4847-af2d-d8e666cb9d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c7c316036c489b8700ff3367dc0835'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:28.971541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 990/1988] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:28 2022] DELETE /identity/v3/users/81c7c316036c489b8700ff3367dc0835 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:29.065122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.068891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:29.090965 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940c93-fcef-4db4-bd93-bbbf0e5647e0 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.260300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.261243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.261727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.262182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] PATH_INFO: `/v3/projects/b338ec63500e45a68fcf31ab48c39cf7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.263542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b338ec63500e45a68fcf31ab48c39cf7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.265744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.349182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b338ec63500e45a68fcf31ab48c39cf7', 'request_id': 'req-9dbcea33-bae8-4fcb-a107-891cd116efc3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.362104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b338ec63500e45a68fcf31ab48c39cf7', 'request_id': 'req-9dbcea33-bae8-4fcb-a107-891cd116efc3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.371944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b338ec63500e45a68fcf31ab48c39cf7', 'request_id': 'req-9dbcea33-bae8-4fcb-a107-891cd116efc3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.388440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] The token cache is being invalidate because project b338ec63500e45a68fcf31ab48c39cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:29.388878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.391062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9dbcea33-bae8-4fcb-a107-891cd116efc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.393275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 999/1989] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:29 2022] DELETE /identity/v3/projects/b338ec63500e45a68fcf31ab48c39cf7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:29.399613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.400808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.401390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.427800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:29.429292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:29.543177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 991/1990] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:29.553199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368aefff-559d-4494-8f41-0963376f402d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.553478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368aefff-559d-4494-8f41-0963376f402d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.553834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368aefff-559d-4494-8f41-0963376f402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.588996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-368aefff-559d-4494-8f41-0963376f402d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:29.590240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-368aefff-559d-4494-8f41-0963376f402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:29.602612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1000/1991] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:29.603800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.614493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb3357-60ff-47c0-af82-6013189a254b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.621535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.622303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.622678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.623603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.624425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:29.624858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.626620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.647296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 992/1992] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:29.651109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368aefff-559d-4494-8f41-0963376f402d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.662172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368aefff-559d-4494-8f41-0963376f402d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.669525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.669685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.670453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.671821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.672966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.676637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.725779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1001/1993] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:29 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:29.731749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.742796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.745864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e14af-5773-4378-ba15-7b1f31735b9f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.747396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:29.754352 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.835073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.857933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d422-9972-42a0-8d07-39584a8fa4e0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.865461 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.866109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.866380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.866644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] PATH_INFO: `/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.868645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dddb9fda3c8442ca0ba8bd61d567cf1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.870574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.875875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.876882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.877311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.878442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.879255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.884655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.928033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 993/1994] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:29 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:29.933499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.954390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dddb9fda3c8442ca0ba8bd61d567cf1', 'request_id': 'req-50faaa4f-b32f-436d-8be5-a617a473ccfb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.959971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54c6e-2dd7-4c8e-8950-720221b28229 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.983960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dddb9fda3c8442ca0ba8bd61d567cf1', 'request_id': 'req-50faaa4f-b32f-436d-8be5-a617a473ccfb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:29.999635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dddb9fda3c8442ca0ba8bd61d567cf1', 'request_id': 'req-50faaa4f-b32f-436d-8be5-a617a473ccfb'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:30.016694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] The token cache is being invalidate because project 4dddb9fda3c8442ca0ba8bd61d567cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:30.016988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:30.018299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50faaa4f-b32f-436d-8be5-a617a473ccfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:30.024465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1002/1995] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:29 2022] DELETE /identity/v3/projects/4dddb9fda3c8442ca0ba8bd61d567cf1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:30.026213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850e4a-238e-4052-8e5b-ddcd49fc893c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.026618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850e4a-238e-4052-8e5b-ddcd49fc893c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.026985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850e4a-238e-4052-8e5b-ddcd49fc893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.051806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fb850e4a-238e-4052-8e5b-ddcd49fc893c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:30.053994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fb850e4a-238e-4052-8e5b-ddcd49fc893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.078766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.079409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.079694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.079968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.080647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.082256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.113071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 994/1996] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:30.115813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.123498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.148455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd74f2d-194d-448e-90e1-0b997aff39b1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.158819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1003/1997] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:30.171819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.171819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.172105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.174551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.175543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.181279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.182770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.184013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.186146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.207782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 995/1998] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:30.210444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:30.212204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.212856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.226280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1004/1999] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:30.227708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.245732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7a225-eb7e-449b-a5f9-289458c632e1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.246332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec06b12-3f24-4d00-a29f-a50b3c0b2837 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.257781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.258417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.258694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.258963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.259604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:30.259910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.271112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.272484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.273174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.273887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.273887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] PATH_INFO: `/v3/projects/b7286fc11529424688d9fdda8c970ccf/users/4868fae6cbc14c0a8bf23d3c7ff4f366/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.305330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1005/2000] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:30.311634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.323621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7286fc11529424688d9fdda8c970ccf, user_id=4868fae6cbc14c0a8bf23d3c7ff4f366, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.326472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daa0c7-7eac-4e1c-b1c6-47db301b8eb7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.335472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde655c5-c9b3-4d79-b5ed-6694f1d0c134 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.342401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.343138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.343409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.343716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.344450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.346859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.380176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 996/2001] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:30 2022] PUT /identity/v3/projects/b7286fc11529424688d9fdda8c970ccf/users/4868fae6cbc14c0a8bf23d3c7ff4f366/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:30.382921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.383216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.383483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.447088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1006/2002] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:30.447435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.463610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] MFA Rules not processed for user `4868fae6cbc14c0a8bf23d3c7ff4f366`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:30.464946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.561827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.581340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.606721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 997/2003] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:30.611404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.611543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ec22-f821-4899-bbf5-9909f5fe54a5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.617137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.618249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.622475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.622769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.623434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.625938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ed090-90e8-4b70-8203-ac3c60d0bd3b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.633009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650e6c-8e56-4056-beaa-852b2880240f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.642668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.643294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.644019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.644019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.644568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.646562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.666391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1007/2004] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:30.667619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.667922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.668184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.687013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 998/2005] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:30.688199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.700514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] MFA Rules not processed for user `a0794e19f43a413db886750b0517d3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:30.701790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.803728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5928f-bf78-43a1-8087-96fca26f35e5 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.810032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1008/2006] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:30.810869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.812427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.813306 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.813712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.814071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.814798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.816504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.825789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd7946-32c1-4147-9b2f-8ac1407a75b6 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.832859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.834590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.834885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.835160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.835826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.836443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 999/2007] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:30.837547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.837858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.841259 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:30.858437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1318c-3d7a-4dfe-b514-9388cb738ef4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.863875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1009/2008] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/users => generated 300 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:30.864940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.865550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.866448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.866819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.867187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.867978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.870122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.877683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3eca5b-5fee-439c-8ba4-1a5a3386290f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.884200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.885709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.885709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.885950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.886783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.888677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.901603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1010/2009] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:30.902451 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.912435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1000/2010] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:30 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:30.912435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.942381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a00c-57fa-40d1-af06-1105f1316a13 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.011787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb07d2-6340-46ef-ad4c-6b4cb5c5f8fc admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.028695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.028695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.029299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.029647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.030046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.030046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.030593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.030702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] PATH_INFO: `/v3/projects/8b72c584a3c34b2abf50dfee02b87123/users/750fa6bba8a14bc081132f46db3e7bbd/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.031335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.033393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.050656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1011/2011] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:31.050656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.053670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b72c584a3c34b2abf50dfee02b87123, user_id=750fa6bba8a14bc081132f46db3e7bbd, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.058300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.059952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c6ce-ce47-4073-9668-92ea7641a644 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.066845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.068155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.068869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.069145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.069787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.071880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22ff64-5c54-41a5-97ca-19ed75b95b56 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.078133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1001/2012] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:30 2022] PUT /identity/v3/projects/8b72c584a3c34b2abf50dfee02b87123/users/750fa6bba8a14bc081132f46db3e7bbd/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.079278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.099559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1012/2013] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/users => generated 292 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:31.100907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31337690-761a-4a68-902f-01ed719188a2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.101327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31337690-761a-4a68-902f-01ed719188a2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.101607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31337690-761a-4a68-902f-01ed719188a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.110844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.134492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.146282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-31337690-761a-4a68-902f-01ed719188a2 None None] MFA Rules not processed for user `750fa6bba8a14bc081132f46db3e7bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:31.147536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-31337690-761a-4a68-902f-01ed719188a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:31.155698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d15b-b2bf-4a18-ac73-e9c78b0bfa46 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.161512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.162119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.162405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.162765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] PATH_INFO: `/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3/users/35c504d61a474fdda58ff1d5046ab3b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.180595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa3e3e21ac344e7b0b866cde92f79b3, user_id=35c504d61a474fdda58ff1d5046ab3b5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.183287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.197493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1002/2014] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:31 2022] PUT /identity/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3/users/35c504d61a474fdda58ff1d5046ab3b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.198325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.220698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1013/2015] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:31.221515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31337690-761a-4a68-902f-01ed719188a2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.232608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31337690-761a-4a68-902f-01ed719188a2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.269910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19f3b-98ec-49bc-8244-daf28894b60b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.275521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.276137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.276517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.276836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.277428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.279039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.292885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.293409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1003/2016] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:31.293519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.293773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.294046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.294268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.294685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.297069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1014/2017] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:31.310609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.310902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf24c-00ec-4347-bf8d-a0bea4d0b119 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.314721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.315334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.315614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.315884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.316548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.318650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.318924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d125f93-6198-4293-bac7-d027ae113801 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.324869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.325565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.325985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.326268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] PATH_INFO: `/v3/projects/87f5261f400a414d9b28fc8ce213b317/users/5fb81b0d881544c6a16fc19dd9ef9a55/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.346767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f5261f400a414d9b28fc8ce213b317, user_id=5fb81b0d881544c6a16fc19dd9ef9a55, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.351184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.356094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1004/2018] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:31.357425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.368322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1015/2019] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:31 2022] PUT /identity/v3/projects/87f5261f400a414d9b28fc8ce213b317/users/5fb81b0d881544c6a16fc19dd9ef9a55/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.369341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.445157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6616d4a-555c-4703-b407-4b300bbd3f2a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.452862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.452862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.452862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.452862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] PATH_INFO: `/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3/users/35c504d61a474fdda58ff1d5046ab3b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.453278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480fabe-dbe6-4011-b199-018a4328f0c0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.455292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa3e3e21ac344e7b0b866cde92f79b3, user_id=35c504d61a474fdda58ff1d5046ab3b5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.458331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.458834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.459420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.459702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.459977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.460653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.463307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac66a6-789a-4883-a75b-eabc4a8065ab admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.480303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1005/2020] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:31 2022] PUT /identity/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3/users/35c504d61a474fdda58ff1d5046ab3b5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.480303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.487765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.495323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1016/2021] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/users => generated 301 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:31.496791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.497046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.497175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.498272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.539600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] MFA Rules not processed for user `35c504d61a474fdda58ff1d5046ab3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:31.541017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:31.595585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9934385-0b11-4e08-aa9a-1969d2d8cbbc admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.602561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.603277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.603744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.604089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.606863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.608138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.628007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1006/2022] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:31.632851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.639705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1017/2023] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:31.639705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.643860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207faf-d551-4b1a-a472-3c691b43d01d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.649482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870702f-70df-4a67-a28b-5ce349119b58 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.650057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.650737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.651088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.651518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.652697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.656878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.657655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.657969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.659072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.659429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] PATH_INFO: `/v3/projects/87f5261f400a414d9b28fc8ce213b317/users/5fb81b0d881544c6a16fc19dd9ef9a55/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.664077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f5261f400a414d9b28fc8ce213b317, user_id=5fb81b0d881544c6a16fc19dd9ef9a55, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.668180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.670653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1007/2024] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:31.671616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.673418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:31.683517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1018/2025] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:31 2022] PUT /identity/v3/projects/87f5261f400a414d9b28fc8ce213b317/users/5fb81b0d881544c6a16fc19dd9ef9a55/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.684268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.746158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c97d9-c66c-4558-872e-f405eee08e6f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.791827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6120688-aab3-414e-b7db-0237bf6cd1ef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.803142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.803672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.805137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.805794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] PATH_INFO: `/v3/projects/7eaf6c5d26174bc19e65437c418b54a7/users/4ba2833f7f8d4691ac89888528fcb93f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.835640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eaf6c5d26174bc19e65437c418b54a7, user_id=4ba2833f7f8d4691ac89888528fcb93f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.838406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.841835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.843363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.843802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.844090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.871633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1019/2026] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:31 2022] PUT /identity/v3/projects/7eaf6c5d26174bc19e65437c418b54a7/users/4ba2833f7f8d4691ac89888528fcb93f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:31.872893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.996500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.002634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.006895 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab5b8-160b-4dc2-b9d3-a8d9602ee3fb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.007460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938553f-46ca-43a3-b5a7-b0c34520be1f service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.013556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.014296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.016700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.016996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.017943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.022555 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.044405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1008/2027] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:32.051194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1020/2028] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:32.053435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.076784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fea5-6617-4d1f-8434-856cc6048d95 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.095929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.096663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.096942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.097259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] PATH_INFO: `/v3/projects/7eaf6c5d26174bc19e65437c418b54a7/users/4ba2833f7f8d4691ac89888528fcb93f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.099869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eaf6c5d26174bc19e65437c418b54a7, user_id=4ba2833f7f8d4691ac89888528fcb93f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.102576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.131172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1021/2029] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:32 2022] PUT /identity/v3/projects/7eaf6c5d26174bc19e65437c418b54a7/users/4ba2833f7f8d4691ac89888528fcb93f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:32.157296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.157751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.158123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.172553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.227899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] MFA Rules not processed for user `4ba2833f7f8d4691ac89888528fcb93f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:32.230628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:32.371867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1009/2030] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:32.430166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.573178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d76403-415b-4122-b77a-24e194c5fe7f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.577306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.577929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.578209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.578481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.596283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.598049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.620420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1022/2031] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:32.987343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.180774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6189d-fa73-45b3-93c0-53511b0ea078 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.191327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.192127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.192922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.193277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] PATH_INFO: `/v3/users/750fa6bba8a14bc081132f46db3e7bbd` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.194933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750fa6bba8a14bc081132f46db3e7bbd)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.198402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.334083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750fa6bba8a14bc081132f46db3e7bbd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.335754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.352305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750fa6bba8a14bc081132f46db3e7bbd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.355203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750fa6bba8a14bc081132f46db3e7bbd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.356057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3fbd0-3242-483d-8163-05e11deceb29 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.363604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.365715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.366531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.367078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] PATH_INFO: `/v3/users/4868fae6cbc14c0a8bf23d3c7ff4f366` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.368167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750fa6bba8a14bc081132f46db3e7bbd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.368997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4868fae6cbc14c0a8bf23d3c7ff4f366)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.369467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750fa6bba8a14bc081132f46db3e7bbd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.371952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.415858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1010/2032] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:32 2022] DELETE /identity/v3/users/750fa6bba8a14bc081132f46db3e7bbd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:33.417303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.423884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.445441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.524776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4868fae6cbc14c0a8bf23d3c7ff4f366'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.526562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4868fae6cbc14c0a8bf23d3c7ff4f366'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.541578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4868fae6cbc14c0a8bf23d3c7ff4f366'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.543007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4868fae6cbc14c0a8bf23d3c7ff4f366'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.550616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148f01-8257-4ea1-bc15-7f571fb1f049 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.554608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c5932189-227a-4998-bd90-633948bb5285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4868fae6cbc14c0a8bf23d3c7ff4f366'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.557417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.558548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.559164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.559591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] PATH_INFO: `/v3/users/b2140cb87fc54cceb72096610a355b5d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.573470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2140cb87fc54cceb72096610a355b5d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.575471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.607679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1023/2033] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:33 2022] DELETE /identity/v3/users/4868fae6cbc14c0a8bf23d3c7ff4f366 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:33.615784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2a8d-4930-4840-bf11-26c7b3919e36 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.616127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2a8d-4930-4840-bf11-26c7b3919e36 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.617266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2a8d-4930-4840-bf11-26c7b3919e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.667155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2140cb87fc54cceb72096610a355b5d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.672123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d76a2a8d-4930-4840-bf11-26c7b3919e36 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:33.673402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d76a2a8d-4930-4840-bf11-26c7b3919e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.683022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2140cb87fc54cceb72096610a355b5d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.683854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2140cb87fc54cceb72096610a355b5d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.693025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2140cb87fc54cceb72096610a355b5d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.694039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f3ffd4c4-f563-4918-b92c-7b4438c4e0c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2140cb87fc54cceb72096610a355b5d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:33.731860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1011/2034] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:33 2022] DELETE /identity/v3/users/b2140cb87fc54cceb72096610a355b5d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:33.735615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.735945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.736230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.795700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:33.797120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.807404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.854035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1024/2035] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:33.855411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f86c-b6c0-443e-809d-429309921720 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.856096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f86c-b6c0-443e-809d-429309921720 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.857987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f86c-b6c0-443e-809d-429309921720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.903039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f926f86c-b6c0-443e-809d-429309921720 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:33.904368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f926f86c-b6c0-443e-809d-429309921720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.914876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1012/2036] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:33.915980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.919216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1025/2037] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:33.925393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f86c-b6c0-443e-809d-429309921720 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.927524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f926f86c-b6c0-443e-809d-429309921720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.931965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:33.975903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.047241 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287b771-9af9-4fed-aab2-1b2a4f2f4fdc None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.055341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.055990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.056291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.056587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.069917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.071825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.073694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f86c-b6c0-443e-809d-429309921720 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.093444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.094085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.094401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.094619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.101499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1013/2038] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:34.103478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.130478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359e7f-43a8-4600-9d51-ac2de0d6b46e service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.136533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.139186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.139349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.139810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.140091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.140401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.165661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.167129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.172888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1026/2039] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:34.200050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1014/2040] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:35.160574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.163078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:35.207877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5166e3-36bb-4058-8d37-785227c51f98 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.217429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.218429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.218895 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.219346 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] PATH_INFO: `/v3/projects/b7286fc11529424688d9fdda8c970ccf` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.221443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7286fc11529424688d9fdda8c970ccf)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.225490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.338002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7286fc11529424688d9fdda8c970ccf', 'request_id': 'req-633669d1-5b90-49f8-9b77-b3de30195ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.354095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7286fc11529424688d9fdda8c970ccf', 'request_id': 'req-633669d1-5b90-49f8-9b77-b3de30195ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.369243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7286fc11529424688d9fdda8c970ccf', 'request_id': 'req-633669d1-5b90-49f8-9b77-b3de30195ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.401535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] The token cache is being invalidate because project b7286fc11529424688d9fdda8c970ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:35.401923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.403367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.409112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.409546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1027/2041] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:35 2022] DELETE /identity/v3/projects/b7286fc11529424688d9fdda8c970ccf => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:35.432784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbfa87-6b2e-4e1c-aebf-0d5583a0281c service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.614768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.615413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.615687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.615956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] PATH_INFO: `/v3/users/35c504d61a474fdda58ff1d5046ab3b5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.618856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c504d61a474fdda58ff1d5046ab3b5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.624462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.749563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.751456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:35.774391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c504d61a474fdda58ff1d5046ab3b5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.782617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633669d1-5b90-49f8-9b77-b3de30195ea9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.791479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c504d61a474fdda58ff1d5046ab3b5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.792142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c504d61a474fdda58ff1d5046ab3b5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.797483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.798375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.798713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.799086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] PATH_INFO: `/v3/projects/b792b66b7d974b65b05c68eeb28c76b7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.802361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b792b66b7d974b65b05c68eeb28c76b7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.817606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c504d61a474fdda58ff1d5046ab3b5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.825580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.825756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c504d61a474fdda58ff1d5046ab3b5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:35.942910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1015/2042] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:35 2022] DELETE /identity/v3/users/35c504d61a474fdda58ff1d5046ab3b5 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:35.943954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.953307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:35.996979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b792b66b7d974b65b05c68eeb28c76b7', 'request_id': 'req-398308df-aba4-45bd-9e2a-c6689acad497'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.012327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b792b66b7d974b65b05c68eeb28c76b7', 'request_id': 'req-398308df-aba4-45bd-9e2a-c6689acad497'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.035327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b792b66b7d974b65b05c68eeb28c76b7', 'request_id': 'req-398308df-aba4-45bd-9e2a-c6689acad497'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.038618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.091677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] The token cache is being invalidate because project b792b66b7d974b65b05c68eeb28c76b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:36.092026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.101420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.107030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1028/2043] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:35 2022] DELETE /identity/v3/projects/b792b66b7d974b65b05c68eeb28c76b7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:36.115793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.121171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:36.156048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.222854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdfcff-ad66-4206-b9e4-a83dd9a08c85 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.239708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.242146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.244647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.244790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] PATH_INFO: `/v3/projects/8b72c584a3c34b2abf50dfee02b87123` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.249350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b72c584a3c34b2abf50dfee02b87123)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.250927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.317634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398308df-aba4-45bd-9e2a-c6689acad497 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.328994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.330312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.332174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.333294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] PATH_INFO: `/v3/users/16d2145b9063476982120121c99884fd` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.337655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d2145b9063476982120121c99884fd)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.346946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.390679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b72c584a3c34b2abf50dfee02b87123', 'request_id': 'req-571c142c-511b-44cd-be1b-2fc4ae206d64'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.408302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b72c584a3c34b2abf50dfee02b87123', 'request_id': 'req-571c142c-511b-44cd-be1b-2fc4ae206d64'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.440885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b72c584a3c34b2abf50dfee02b87123', 'request_id': 'req-571c142c-511b-44cd-be1b-2fc4ae206d64'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.497542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d2145b9063476982120121c99884fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.498895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] The token cache is being invalidate because project 8b72c584a3c34b2abf50dfee02b87123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:36.499194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.503296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d2145b9063476982120121c99884fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.514318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.516446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1016/2044] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:35 2022] DELETE /identity/v3/projects/8b72c584a3c34b2abf50dfee02b87123 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:36.523953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.525450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:36.529232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d2145b9063476982120121c99884fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.530412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d2145b9063476982120121c99884fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.553304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d2145b9063476982120121c99884fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:36.636986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1029/2045] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:36 2022] DELETE /identity/v3/users/16d2145b9063476982120121c99884fd => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:36.771183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c142c-511b-44cd-be1b-2fc4ae206d64 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.927511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.929520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:36.965341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.966000 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.969903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.970253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] PATH_INFO: `/v3/users/4ba2833f7f8d4691ac89888528fcb93f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.972169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba2833f7f8d4691ac89888528fcb93f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.975007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.043888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:37.164698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba2833f7f8d4691ac89888528fcb93f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.182855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba2833f7f8d4691ac89888528fcb93f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.183648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba2833f7f8d4691ac89888528fcb93f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.202536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba2833f7f8d4691ac89888528fcb93f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.203981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ad54a-2223-4e72-9429-5cc9800b7c2f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.211902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-203e88e3-a49e-4198-baa1-2d605b83c6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba2833f7f8d4691ac89888528fcb93f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.221668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.222637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.223312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.224519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] PATH_INFO: `/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.226792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa3e3e21ac344e7b0b866cde92f79b3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.241033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.271232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1017/2046] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:36 2022] DELETE /identity/v3/users/4ba2833f7f8d4691ac89888528fcb93f => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:37.298764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:37.386928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa3e3e21ac344e7b0b866cde92f79b3', 'request_id': 'req-84e5b17f-89af-4c65-b419-12088a035ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.408607 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa3e3e21ac344e7b0b866cde92f79b3', 'request_id': 'req-84e5b17f-89af-4c65-b419-12088a035ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.423856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa3e3e21ac344e7b0b866cde92f79b3', 'request_id': 'req-84e5b17f-89af-4c65-b419-12088a035ebe'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.459724 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] The token cache is being invalidate because project 5fa3e3e21ac344e7b0b866cde92f79b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:37.460855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.461776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84e5b17f-89af-4c65-b419-12088a035ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:37.467229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1030/2047] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:36 2022] DELETE /identity/v3/projects/5fa3e3e21ac344e7b0b866cde92f79b3 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:37.472134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.473080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.473590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.512732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:37.515081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:37.681686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1018/2048] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:37.684110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.684447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.684725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.757660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:37.759741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:37.812719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1031/2049] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:37.813946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:37.839228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbce10a-9f81-4d01-a1d5-f827b367376c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.868647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.869691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.869992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.870242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.870929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:37.871281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.876447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.936625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1019/2050] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:37.939084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:37.957479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebea366-92de-40d9-8818-5ae0904511f7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.983166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.986246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.986630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.987012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.987933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.991059 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.041325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1032/2051] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:37 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:38.046964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.064037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d06b-cd68-44c8-8d77-3e82e3940d77 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.095447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.098124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:38.181490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.182932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.183946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.184374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.185219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.188391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.224570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1020/2052] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:38 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:38.228698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.244092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6df23-9e65-42ee-842c-1c766c054302 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.253636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.255800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.258120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.258634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.259567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.261456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.283227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:38.307975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1021/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:38.313177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.333408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d8a8-2980-4154-a8b4-0d63d0df1654 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.336183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8e5b-c73b-45f7-8dfe-e185dc452af9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.354054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.354707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.354993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.355519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.356713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.360081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.362021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.362762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.363050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.363381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] PATH_INFO: `/v3/projects/08c3961ab4044c35b585b1b52c4e80f9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.366907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c3961ab4044c35b585b1b52c4e80f9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.370927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.389790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1022/2054] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:38.393721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.408229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6655fe-dcfc-4d94-8fee-48684dcae5b3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.449894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.450580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.453268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.453570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] PATH_INFO: `/v3/projects/ea594a5db4d54a589f3d308aa8cdb4e7/users/825af561e30446589faeb0d3afa23e52/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.508542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea594a5db4d54a589f3d308aa8cdb4e7, user_id=825af561e30446589faeb0d3afa23e52, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.513435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046b0b3-d418-4489-bf0c-58b7d6fa13c0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.562109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1023/2055] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:38 2022] PUT /identity/v3/projects/ea594a5db4d54a589f3d308aa8cdb4e7/users/825af561e30446589faeb0d3afa23e52/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:38.569783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22614-8aab-4358-b71e-89a73fbc643e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.570516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22614-8aab-4358-b71e-89a73fbc643e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.575896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22614-8aab-4358-b71e-89a73fbc643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.585839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c3961ab4044c35b585b1b52c4e80f9', 'request_id': 'req-ba915448-51cd-4522-966f-47f587b1a47e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:38.603620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c3961ab4044c35b585b1b52c4e80f9', 'request_id': 'req-ba915448-51cd-4522-966f-47f587b1a47e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:38.616324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c3961ab4044c35b585b1b52c4e80f9', 'request_id': 'req-ba915448-51cd-4522-966f-47f587b1a47e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:38.644912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] The token cache is being invalidate because project 08c3961ab4044c35b585b1b52c4e80f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:38.646609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:38.649565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:38.656121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-25c22614-8aab-4358-b71e-89a73fbc643e None None] MFA Rules not processed for user `825af561e30446589faeb0d3afa23e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:38.663858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1033/2056] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:38 2022] DELETE /identity/v3/projects/08c3961ab4044c35b585b1b52c4e80f9 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:38.665081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.666844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-25c22614-8aab-4358-b71e-89a73fbc643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:38.668333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:38.681504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:38.831471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba915448-51cd-4522-966f-47f587b1a47e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.852661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1024/2057] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:38.860120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.862902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.865440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.913037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:38.916194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.024415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.025259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.025649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.026078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] PATH_INFO: `/v3/projects/7eaf6c5d26174bc19e65437c418b54a7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.030383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eaf6c5d26174bc19e65437c418b54a7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.035011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.145775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eaf6c5d26174bc19e65437c418b54a7', 'request_id': 'req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:39.150714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1025/2058] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:39.154997 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.157166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eaf6c5d26174bc19e65437c418b54a7', 'request_id': 'req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:39.158509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.167128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eaf6c5d26174bc19e65437c418b54a7', 'request_id': 'req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:39.185988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] The token cache is being invalidate because project 7eaf6c5d26174bc19e65437c418b54a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:39.186534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:39.186705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dda03-dc92-426f-a287-1550251f4a09 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.187606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e19e42d3-f0ac-4fef-a3dc-6c21e74e75d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:39.191694 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1034/2059] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:38 2022] DELETE /identity/v3/projects/7eaf6c5d26174bc19e65437c418b54a7 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:39.194512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e46df-6ba9-4d69-b55f-806abdabdcb7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.195295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e46df-6ba9-4d69-b55f-806abdabdcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.195738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e46df-6ba9-4d69-b55f-806abdabdcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.240577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ce0e46df-6ba9-4d69-b55f-806abdabdcb7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:39.243983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ce0e46df-6ba9-4d69-b55f-806abdabdcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.324686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:39.350531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.351185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.351485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.351738 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.352416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.353008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:39.355127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.412855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1035/2060] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:39.412855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.412855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.412855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.457329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:39.460754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.479501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1026/2061] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/projects => generated 383 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:39.480262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.491576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1036/2062] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:39.492896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.498460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.528176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c0f50-1b4c-4536-b033-c57b22e926d9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.653246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3400e8f-a4c6-4f58-aaa6-682369bcc4d9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.653811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.654227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.654227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.654455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.655095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.657914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.663162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.664668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.669772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.670312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.671440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:39.673163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.684305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7c99c-d5f9-4858-ba00-64d0a94a8a6a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.716890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1027/2063] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:39.718264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.718543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.718815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.725259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1037/2064] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:39.729012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.761223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac02c3-6087-4e7b-9110-87906556a357 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.761717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:39.763233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:39.776349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.777410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.777410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.777410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.778053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.784007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.800893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1028/2065] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:39.800893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.822768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b4f65-cf63-4361-82ee-9189f3bec1f3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.839284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.840104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.840404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.840675 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.841454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.845851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.869200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1038/2066] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:39.870532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.884743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1029/2067] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:39.885436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.912413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ba99e-ae3f-4957-94aa-250fcb6636d5 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.051128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166df6b-744d-4c36-94b6-3ea450d8dad9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.079395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.081760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.082138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.082513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.083307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.084114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:40.084582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.084792 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.085455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.086110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.087122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.089599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.089735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.136393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1039/2068] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:40.137228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.156416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1030/2069] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:40.160923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.161429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609bf89-fa05-4c48-9e30-a92824cbf872 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.167345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.168205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.168553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.169703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] PATH_INFO: `/v3/projects/065f137ab27c4e64892c75ee6df5a5a3/users/c8800ca80f4848869d1954d2ac865453/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.197776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272bf00-5176-4f92-abb7-b45c156782db admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.216764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] RBAC: Authorizing `identity:create_grant(project_id=065f137ab27c4e64892c75ee6df5a5a3, user_id=c8800ca80f4848869d1954d2ac865453, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.217016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.217308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.217543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.217831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.218463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.220418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.224209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.265437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1040/2070] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:40 2022] PUT /identity/v3/projects/065f137ab27c4e64892c75ee6df5a5a3/users/c8800ca80f4848869d1954d2ac865453/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:40.266318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.319444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1031/2071] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:40 2022] POST /identity/v3/projects => generated 359 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:40.321917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.329142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.396674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.416635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882141ad-588b-4efe-bdde-b0e7c36c914f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.504844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.506141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.506625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.507815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.508826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.511629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.523274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34981420-80fc-454f-b4c9-d3e9002cf343 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.530956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1041/2072] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:40.531967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.533954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.534740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.534856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.536192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.536906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.538950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.544872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c468e-74b5-41be-a999-ba1c6a0f864b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.563517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.564763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.566614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.569844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.572722 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1032/2073] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:40.573888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.576106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.585909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.598902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fb741-b7f8-4eb9-871b-c28eece27f83 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.608274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.609342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.609806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.610186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.612945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.616935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.632545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1042/2074] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:40 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:40.633303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.637851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1033/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:40.639609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.649214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a9d0-f6df-44e1-a0ff-5b9cd769da5e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.656506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.657184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.657612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.657893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] PATH_INFO: `/v3/projects/065f137ab27c4e64892c75ee6df5a5a3/users/c8800ca80f4848869d1954d2ac865453/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.659866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=065f137ab27c4e64892c75ee6df5a5a3, user_id=c8800ca80f4848869d1954d2ac865453, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.663102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5fcdb-bb72-41de-91b6-a63f331f9e7f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.663625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.678776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.679579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.679863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.680858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.681523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.684609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bac45-e508-4b68-b083-5e5482a29790 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.701308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1043/2076] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:40 2022] PUT /identity/v3/projects/065f137ab27c4e64892c75ee6df5a5a3/users/c8800ca80f4848869d1954d2ac865453/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:40.702593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1034/2077] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:40.704559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.706785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a752f6-2a58-4893-9dad-60bbd5387230 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.707074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a752f6-2a58-4893-9dad-60bbd5387230 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.707343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a752f6-2a58-4893-9dad-60bbd5387230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.812528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-20a752f6-2a58-4893-9dad-60bbd5387230 None None] MFA Rules not processed for user `c8800ca80f4848869d1954d2ac865453`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:40.819250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-20a752f6-2a58-4893-9dad-60bbd5387230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:40.910298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c93c2-ea7a-419b-a924-90c28db2a5be admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.925341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.927873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.936230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.936230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] PATH_INFO: `/v3/projects/77ab964cb35e4083b86a4b92d8853ff9/users/58b4b477e5b146b19a2300114b90230f/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.997233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ab964cb35e4083b86a4b92d8853ff9, user_id=58b4b477e5b146b19a2300114b90230f, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.002328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.031281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1044/2078] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:40 2022] PUT /identity/v3/projects/77ab964cb35e4083b86a4b92d8853ff9/users/58b4b477e5b146b19a2300114b90230f/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:41.042736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.056064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1035/2079] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:41.058043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9b21-a327-488d-9693-6d170dc3a2ef None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.058043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9b21-a327-488d-9693-6d170dc3a2ef None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.058191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9b21-a327-488d-9693-6d170dc3a2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.080481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.128990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e6ab9b21-a327-488d-9693-6d170dc3a2ef None None] MFA Rules not processed for user `58b4b477e5b146b19a2300114b90230f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:41.132031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e6ab9b21-a327-488d-9693-6d170dc3a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.208199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393b9-7590-4a39-aafd-23772266ee3d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.223839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.226348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.226776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.227170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.232628 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.234646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.260154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1045/2080] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:41.262487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.266795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.307362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1036/2081] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:41.308834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.309282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.309670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.345253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.350502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] MFA Rules not processed for user `5fb81b0d881544c6a16fc19dd9ef9a55`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:41.352524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.417840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeff90-11da-4c02-b8b7-4536fff11a59 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.423655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.425414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.428295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.428295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.431551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.479807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1037/2082] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:41.480952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.493547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae969213-f89f-4974-993a-0f2d7c20c783 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.499385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.500463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.501064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.501499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] PATH_INFO: `/v3/projects/76862b7a965041eba3e19e7cecd13c5d/users/ad345b9083e3492b9e3b81c1dae4a464/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.530232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76862b7a965041eba3e19e7cecd13c5d, user_id=ad345b9083e3492b9e3b81c1dae4a464, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.538074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.563178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1038/2083] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:41 2022] PUT /identity/v3/projects/76862b7a965041eba3e19e7cecd13c5d/users/ad345b9083e3492b9e3b81c1dae4a464/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:41.564052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.586527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.587983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.604035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1046/2084] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:41.606006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.620433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.737604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369012b-3d96-46f5-b1bf-18ec42eae451 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.747189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.748323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.748880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.749806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.751111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.754030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29062b-faa8-4562-bde9-94b9e21fccd6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.767937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.795024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452826-c917-436b-9800-418cd6f2afae service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.835116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1039/2085] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/projects => generated 387 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:41.838361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.838690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.838994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.909899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] MFA Rules not processed for user `ad345b9083e3492b9e3b81c1dae4a464`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:41.911463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:42.035070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.038074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.038074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.038074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.044009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:42.094320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:42.143372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1040/2086] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:42.144793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.273219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.276944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.302073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1047/2087] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:42.305862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.328459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4d77a-e71b-47dd-a600-193d64159660 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.340494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.343734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.345194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.345641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.347930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.353984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.416100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1041/2088] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:42 2022] POST /identity/v3/users => generated 300 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:42.417384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.443925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a17357-7df5-4ef3-a13b-b4ce2f62cb24 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.468514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370204a3-e4d4-4b33-b924-468131eefebe service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.468742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.468742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.468742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.468742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.469181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.473203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.498490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1042/2089] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:42.503136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.526671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7fb5-8062-4df4-8a04-8569b9e52f77 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.543190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.544755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.544755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.544755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] PATH_INFO: `/v3/projects/f3a6266485a6471da09c2e754b92e590/users/0dc2c5ad78614742816cb8e3c5c7522c/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.600036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a6266485a6471da09c2e754b92e590, user_id=0dc2c5ad78614742816cb8e3c5c7522c, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.607252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.650457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1043/2090] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:42 2022] PUT /identity/v3/projects/f3a6266485a6471da09c2e754b92e590/users/0dc2c5ad78614742816cb8e3c5c7522c/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:42.661404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.662226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.662575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.662920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.665424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.675948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:42.680620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.684168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.717843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1048/2091] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:42.739212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.764576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c13b46-2212-4b6b-98ee-838bafc8a8a0 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.913024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.914085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.914184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.914438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] PATH_INFO: `/v3/users/ad345b9083e3492b9e3b81c1dae4a464` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.916986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a735-41e7-4dd7-9b99-8c8f91646241 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.917484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad345b9083e3492b9e3b81c1dae4a464)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.921938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.931854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.932537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.932816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.933086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.935999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.940894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.980618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1044/2092] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:42.985278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.011406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff874a-404d-4646-835e-388e03403074 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.025798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.026938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.027516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.027648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad345b9083e3492b9e3b81c1dae4a464'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.027803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] PATH_INFO: `/v3/projects/f3a6266485a6471da09c2e754b92e590/users/0dc2c5ad78614742816cb8e3c5c7522c/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.029489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad345b9083e3492b9e3b81c1dae4a464'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.034266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a6266485a6471da09c2e754b92e590, user_id=0dc2c5ad78614742816cb8e3c5c7522c, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.040353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a4ad-1483-4e25-8e18-ae112d763d01 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.047270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad345b9083e3492b9e3b81c1dae4a464'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.048648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad345b9083e3492b9e3b81c1dae4a464'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.065730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fb9f32c4-a1ef-424e-845d-15680f18d1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad345b9083e3492b9e3b81c1dae4a464'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.083835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1045/2093] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:42 2022] PUT /identity/v3/projects/f3a6266485a6471da09c2e754b92e590/users/0dc2c5ad78614742816cb8e3c5c7522c/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:43.084070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.084374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.084650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.136424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1049/2094] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:42 2022] DELETE /identity/v3/users/ad345b9083e3492b9e3b81c1dae4a464 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:43.141077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.141267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.141545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.165483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] MFA Rules not processed for user `0dc2c5ad78614742816cb8e3c5c7522c`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:43.166702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.173437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:43.175328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.189096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.324130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1046/2095] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:43.336957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.339483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.341299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1050/2096] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:43.346595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.350455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.406143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.451658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179627b-8c8d-4e55-bacc-38214eebef1f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.472494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.472837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.475695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.475982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.503077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107416b-6546-474f-870b-8ff271d0cb2a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.511470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.512785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.513300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.514436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.516078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.517679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca39bc-b2c5-4044-9bc1-47fda41741dd service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.531804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.539048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.555840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1051/2097] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:43.572264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1047/2098] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:43.666842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.669975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.694373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2662b-0357-487f-b613-8d2759838842 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.704902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.705927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.706361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.706812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] PATH_INFO: `/v3/users/58b4b477e5b146b19a2300114b90230f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.710996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b4b477e5b146b19a2300114b90230f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.713365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.849531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b4b477e5b146b19a2300114b90230f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.851518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b4b477e5b146b19a2300114b90230f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.869063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b4b477e5b146b19a2300114b90230f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.869809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b4b477e5b146b19a2300114b90230f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.896534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b4b477e5b146b19a2300114b90230f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:43.961637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1052/2099] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:43 2022] DELETE /identity/v3/users/58b4b477e5b146b19a2300114b90230f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:43.965919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.966381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.966501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.999408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:44.000629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:44.193930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1048/2100] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:44.219649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.221549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:44.437275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d0005-41fb-43f5-8fb6-5948b38a5a3a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.472631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.473580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.473884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.474266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.543493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.543876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.545366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254dce3-c1ff-4e54-a5a2-e0478dd53d6f service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.545497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:44.572503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1053/2101] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:44.581134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41505-6d44-4e7c-b84f-32b1e0368fb0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.613861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.614528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.614813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.615078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] PATH_INFO: `/v3/projects/76862b7a965041eba3e19e7cecd13c5d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.621184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76862b7a965041eba3e19e7cecd13c5d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.625272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.750229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76862b7a965041eba3e19e7cecd13c5d', 'request_id': 'req-8fc33f25-13ca-447b-9e8b-f17a98a11e88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:44.752375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:44.762029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76862b7a965041eba3e19e7cecd13c5d', 'request_id': 'req-8fc33f25-13ca-447b-9e8b-f17a98a11e88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:44.781665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76862b7a965041eba3e19e7cecd13c5d', 'request_id': 'req-8fc33f25-13ca-447b-9e8b-f17a98a11e88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:44.810828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] The token cache is being invalidate because project 76862b7a965041eba3e19e7cecd13c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:44.811434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:44.812497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8fc33f25-13ca-447b-9e8b-f17a98a11e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:44.818839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1049/2102] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:44 2022] DELETE /identity/v3/projects/76862b7a965041eba3e19e7cecd13c5d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:44.824669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.825207 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.828442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.871366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:44.874229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:44.977966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.067604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba827a-a56e-4928-a094-26914c19989a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.068586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba827a-a56e-4928-a094-26914c19989a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.069012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba827a-a56e-4928-a094-26914c19989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.080622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1054/2103] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:45.102061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-26ba827a-a56e-4928-a094-26914c19989a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:45.103880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-26ba827a-a56e-4928-a094-26914c19989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:45.124090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.127343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1050/2104] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:45.142129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacf443-0808-4c66-8542-348e58140e7c None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.146939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.147758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.148481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.149596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.150156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:45.150776 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.152309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.179680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1055/2105] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:45.184831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba827a-a56e-4928-a094-26914c19989a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.201393 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba827a-a56e-4928-a094-26914c19989a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.211988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.213208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.213208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.213530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.214922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.217892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.281536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1051/2106] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:45 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:45.281749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.297212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8829d-404e-42df-b9bb-b3798f1c65c9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.396473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.396473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.396473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.398834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.399631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.402043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.447359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1056/2107] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:45 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:45.447829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.468049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9f70-cae9-4fd8-aa78-4d6674c0db13 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.476690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.478768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.479267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.480832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.485822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.487603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.511822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.523408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1052/2108] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:45.525396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6038a5b-4e5d-4500-a062-a5dcec81622d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.532052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.533265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.533640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.534005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.535335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.538193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f24869-d514-4c62-982c-1a80a5762141 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.558444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1057/2109] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:45.559647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.573489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4bbc1-fa1c-4d8f-9826-8deaffa12cd4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.589768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.595195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.598662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.599342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] PATH_INFO: `/v3/projects/241e5470186a4c93b2465ecc84d80603/users/1e391d2591734de3bbe2515c703c4602/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.642289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] RBAC: Authorizing `identity:create_grant(project_id=241e5470186a4c93b2465ecc84d80603, user_id=1e391d2591734de3bbe2515c703c4602, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.650960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.693848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba397e0-905b-4428-8564-dfa0520d0262 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.694959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba397e0-905b-4428-8564-dfa0520d0262 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.695319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba397e0-905b-4428-8564-dfa0520d0262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.696971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1053/2110] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:45 2022] PUT /identity/v3/projects/241e5470186a4c93b2465ecc84d80603/users/1e391d2591734de3bbe2515c703c4602/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:45.759447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-aba397e0-905b-4428-8564-dfa0520d0262 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:45.763053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-aba397e0-905b-4428-8564-dfa0520d0262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:45.941523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1058/2111] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:45.954899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.960791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:46.153766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b72db-3412-4148-ada7-0c42b462504f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.167138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.167768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.168045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.168373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.184946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.186606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678bf42-9314-4d6c-a46c-5ac6a74a6105 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.241283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1054/2112] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:47.058402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.059316 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.059597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.061066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1059/2113] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:30:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:47.101732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.103219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.103720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.142171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:47.145009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.160009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.163580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.222452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39f1c-030d-4884-85f9-6616e5908909 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.246008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.246666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.247614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.247910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] PATH_INFO: `/v3/projects/77ab964cb35e4083b86a4b92d8853ff9` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.249381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ab964cb35e4083b86a4b92d8853ff9)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.255414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.423413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.450358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ab964cb35e4083b86a4b92d8853ff9', 'request_id': 'req-25a31b08-888a-41b3-a798-cf839637b10a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.451067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1055/2114] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:30:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:47.452327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.457630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.463182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ab964cb35e4083b86a4b92d8853ff9', 'request_id': 'req-25a31b08-888a-41b3-a798-cf839637b10a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.472003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ab964cb35e4083b86a4b92d8853ff9', 'request_id': 'req-25a31b08-888a-41b3-a798-cf839637b10a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.509307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] The token cache is being invalidate because project 77ab964cb35e4083b86a4b92d8853ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:47.509637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.513506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc309c-48e8-4fd6-aed8-676d59f19276 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.516559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.521825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1060/2115] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:47 2022] DELETE /identity/v3/projects/77ab964cb35e4083b86a4b92d8853ff9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:47.525285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.527343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.541523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.691631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.694679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.695249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.696685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] PATH_INFO: `/v3/users/4056ea3d1a1d48f2a34e14df77914363` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.708149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4056ea3d1a1d48f2a34e14df77914363)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.721043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.750308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b08-888a-41b3-a798-cf839637b10a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.788513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.789081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.789352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.789621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.816294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.859553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.866693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.870903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4056ea3d1a1d48f2a34e14df77914363'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.896586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1061/2116] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:47.897201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4056ea3d1a1d48f2a34e14df77914363'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.897841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4056ea3d1a1d48f2a34e14df77914363'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.908351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4056ea3d1a1d48f2a34e14df77914363'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.909278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4056ea3d1a1d48f2a34e14df77914363'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:47.914592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.919945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.983073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1056/2117] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:47 2022] DELETE /identity/v3/users/4056ea3d1a1d48f2a34e14df77914363 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:47.984403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.986190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.986655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec7320-26ed-4969-95e9-5b3c9a50caad service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.106571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4210768-de2a-4ade-b433-1d9ba0c1d892 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.114590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.116685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.117137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.117552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.126171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:48.153594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.155462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.158613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.158952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] PATH_INFO: `/v3/users/0dc2c5ad78614742816cb8e3c5c7522c` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.161043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc2c5ad78614742816cb8e3c5c7522c)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.163050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:48.234573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:48.263875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.266686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d280d73-c526-431a-8213-a53c3333a73b service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:48.284373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1057/2118] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:48.286978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.288084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.288623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.373486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] MFA Rules not processed for user `a0794e19f43a413db886750b0517d3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:48.379345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:48.385285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc2c5ad78614742816cb8e3c5c7522c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:48.388089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc2c5ad78614742816cb8e3c5c7522c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:48.422991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc2c5ad78614742816cb8e3c5c7522c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:48.425122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc2c5ad78614742816cb8e3c5c7522c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:48.427401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:48.445633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-afa161ea-7154-49a8-aaf0-f965d04b0167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc2c5ad78614742816cb8e3c5c7522c'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:48.489847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1062/2119] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:47 2022] DELETE /identity/v3/users/0dc2c5ad78614742816cb8e3c5c7522c => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:48.491494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.491769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.492014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.588813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] MFA Rules not processed for user `1e391d2591734de3bbe2515c703c4602`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:48.608497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1058/2120] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:48.613848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:48.619394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:48.619715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:48.677278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4bbc-0c29-4e4c-a797-666b51e31fba None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.899393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1063/2121] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:48.913536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.914162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.914542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.914717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] PATH_INFO: `/v3/projects/0a676eb5cbd74956ba12b1d5595e3f18` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.920939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a676eb5cbd74956ba12b1d5595e3f18)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.936123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.074289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.077180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.088856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a676eb5cbd74956ba12b1d5595e3f18', 'request_id': 'req-7fb05984-4dd4-4bf1-ac5e-752708f3da22'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.103558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a676eb5cbd74956ba12b1d5595e3f18', 'request_id': 'req-7fb05984-4dd4-4bf1-ac5e-752708f3da22'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.122140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a676eb5cbd74956ba12b1d5595e3f18', 'request_id': 'req-7fb05984-4dd4-4bf1-ac5e-752708f3da22'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.150600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] The token cache is being invalidate because project 0a676eb5cbd74956ba12b1d5595e3f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:49.150945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.152084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7fb05984-4dd4-4bf1-ac5e-752708f3da22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.153661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1059/2122] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:48 2022] DELETE /identity/v3/projects/0a676eb5cbd74956ba12b1d5595e3f18 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:49.160075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.160736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.161127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.197670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:49.199621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.202359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00f3af-7caf-4b7f-95ed-59c7051856d1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.230137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.275509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.276231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.276542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.276969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.279655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.297774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1060/2123] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:49.298749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.300051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.315085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.316064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84d7e7-4391-4dac-9ca0-cf329318a9c4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.324753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.325574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.325805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.326234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] PATH_INFO: `/v3/projects/f3a6266485a6471da09c2e754b92e590` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.328148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a6266485a6471da09c2e754b92e590)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.329890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.355645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.357952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9d8d9-9bac-4627-a9e8-4dcb6320206b service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.380357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1064/2124] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:49.380357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.380357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.380357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.403141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:49.404392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.413309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a6266485a6471da09c2e754b92e590', 'request_id': 'req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.421960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a6266485a6471da09c2e754b92e590', 'request_id': 'req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.451208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a6266485a6471da09c2e754b92e590', 'request_id': 'req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.466066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] The token cache is being invalidate because project f3a6266485a6471da09c2e754b92e590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:49.466367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.467277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-053a73cc-30a6-45c2-9005-5ebe3ea26c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:49.469138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1061/2125] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:49 2022] DELETE /identity/v3/projects/f3a6266485a6471da09c2e754b92e590 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:49.474623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae704b-b17c-4657-8a5c-0fefe00acf50 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.475550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae704b-b17c-4657-8a5c-0fefe00acf50 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.475948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae704b-b17c-4657-8a5c-0fefe00acf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.497464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1065/2126] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:49.502232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.508010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7dae704b-b17c-4657-8a5c-0fefe00acf50 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:49.509652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7dae704b-b17c-4657-8a5c-0fefe00acf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.515930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a0155-ff61-474d-b10f-bbe6b6d4982a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.519866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1062/2127] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:49.526968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.527756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.527995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.529742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.530021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.530281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.532467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.534699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:49.535975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.539317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.551810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:49.553050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:49.556194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1066/2128] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:49.557092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.562078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1063/2129] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:49.566413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.568955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33107d-8a14-4fb5-9ac5-6a3eb750b8d5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.574678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.575481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.575764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.576032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.576745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.578332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78905f39-ca0a-4706-b7a0-9316fe894190 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.578756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.584391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.585340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.585630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.585915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.586551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:49.586856 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.588665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.606456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1064/2130] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:49.611782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.613077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1067/2131] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:49.614548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.624622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770744-130d-4a48-9028-556a0dc71e49 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.689203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54e13-ccd9-4bb1-af26-a1b7badc4932 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.692288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.693094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.693579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.694501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.694636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.694735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.694811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.695039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.695248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.695873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.696384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.697862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.721657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.737618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1068/2132] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:49.737794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1065/2133] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:49.739959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.740383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.763528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071390-0369-49ea-930a-2c3484ca9d5d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.819012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.827543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda378-cd7f-4c14-997d-9d27e211752a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.835075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.835876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.838526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.839057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.839943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.842173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.870739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1066/2134] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:49.874807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.877353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.877972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.878273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.878548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.879249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.881155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.884735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b050-6ceb-480a-95aa-861461377a97 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.889384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.889981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.890263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.890541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.891180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.894391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1069/2135] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:49.895048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.896809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.903670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1067/2136] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:49.906033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28150f6-454e-40fb-b0b4-61e9ec9148b4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.907419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.911649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.914843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.915131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.915409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.916071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.917698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.919373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f2d7-6f4f-45d9-8431-cd2b70b635f2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.926143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.927531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.927923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.928311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.928972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.930554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.934015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1070/2137] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:49.938120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.942011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1068/2138] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:49.945759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.948587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faa09-ee86-48a2-8421-85f5fcf09c40 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.953747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.954384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.954658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.954932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] PATH_INFO: `/v3/projects/49fe69e57e744eea93988aecfa9decc2/users/30ccb163613a40ac8042f229199717f6/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.955386 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb2a5-11cb-40f5-aa44-66e8e58608ce admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.960910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.961549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.961843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.962133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] PATH_INFO: `/v3/projects/7679a6dc178f4b12a6af2717cbfea33e/users/9b6d6dbc6860401facadf1bc0365483b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.970425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fe69e57e744eea93988aecfa9decc2, user_id=30ccb163613a40ac8042f229199717f6, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.973732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fa8bc-ebad-4888-9e05-eca75bbecfd1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.978309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7679a6dc178f4b12a6af2717cbfea33e, user_id=9b6d6dbc6860401facadf1bc0365483b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.981970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ea3ee-108e-4ac0-b559-31959a19a590 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.989388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1071/2139] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:49 2022] PUT /identity/v3/projects/49fe69e57e744eea93988aecfa9decc2/users/30ccb163613a40ac8042f229199717f6/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:49.995344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.995637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.995892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.001139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1069/2140] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:49 2022] PUT /identity/v3/projects/7679a6dc178f4b12a6af2717cbfea33e/users/9b6d6dbc6860401facadf1bc0365483b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:50.013187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.013470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.013721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.023787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:50.025261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:50.054179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] MFA Rules not processed for user `9b6d6dbc6860401facadf1bc0365483b`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:50.055847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:50.167874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1070/2141] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:30:50.169414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.179744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1072/2142] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:50.184444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e8037-3f63-4d28-84d4-21c2c215735b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.190468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.194075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:50.201766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.202403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.202688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.202966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.203611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.205613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.264021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1071/2143] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:50.267975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.288842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:50.335444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae146f-dbc4-49c3-bd23-a6f7e0ccc5f8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.373217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde676-07ae-4821-887c-17a5ffe303bd admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.384419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.384973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.385251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.385519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.386292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.388157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.394808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:50.413096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.413827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.414875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.414875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.419207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1072/2144] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:50.423224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.428659 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.430139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.437760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e64db0-536c-4a06-b8c4-a76c531e286c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.442709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1073/2145] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:50.449811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.452325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.453104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.453760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.454508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.456295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.470280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1073/2146] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:50.478227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.489874 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca4471-13a1-432f-8dce-5379d39d6708 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.494806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.495497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.495774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.496070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] PATH_INFO: `/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627/users/5c10381e66ee4d929a140ae02b1051ad/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.513532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c34b709c40c4f98a4a6d15e6ce71627, user_id=5c10381e66ee4d929a140ae02b1051ad, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.531108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.571878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1074/2147] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:50 2022] PUT /identity/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627/users/5c10381e66ee4d929a140ae02b1051ad/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:50.576952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.606550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d1d7-be8c-4d03-b168-7cc609e8661b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.711666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.712420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.712680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.713018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.713791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.715411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4260d8b-eef0-4ea4-9103-c31904f1b1a4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.732649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1074/2148] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:50.734225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.754490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2ba63-dadd-4a36-a245-3df2e47a584b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.760755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.761419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.761696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.761967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] PATH_INFO: `/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627/users/5c10381e66ee4d929a140ae02b1051ad/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.764035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c34b709c40c4f98a4a6d15e6ce71627, user_id=5c10381e66ee4d929a140ae02b1051ad, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.766855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf655e6-6fcd-4c27-9614-9cb20876eb7f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.796762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1075/2149] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:50 2022] PUT /identity/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627/users/5c10381e66ee4d929a140ae02b1051ad/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:50.797699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.797991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.798271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.835407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] MFA Rules not processed for user `5c10381e66ee4d929a140ae02b1051ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:50.836679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:50.871417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.872990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.872990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.902449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] MFA Rules not processed for user `1e391d2591734de3bbe2515c703c4602`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:50.904061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:50.980299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1075/2150] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:50.980520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1076/2151] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:50.993482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.995110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:51.012769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:51.075155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fbb59-f2bf-49a9-8400-a52e0ec460d6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.087041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.087744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.088104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.088396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.099275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.100884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.112339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1076/2152] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:51.227854 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.238608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3793b-950e-48f0-8559-b5b4fec06c3e None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.242774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.243400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.243675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.243945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.343815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.345422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775671e-0939-49e8-9d4f-fba09a0ad629 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.357433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1077/2153] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:51.786172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.908834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102cdd4-4289-4783-857c-78b232517e35 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.922208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.923600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.923993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.925028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] PATH_INFO: `/v3/users/9b6d6dbc6860401facadf1bc0365483b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.927372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6d6dbc6860401facadf1bc0365483b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.937396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.076870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6d6dbc6860401facadf1bc0365483b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:52.096210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6d6dbc6860401facadf1bc0365483b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:52.097058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6d6dbc6860401facadf1bc0365483b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:52.112947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6d6dbc6860401facadf1bc0365483b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:52.114757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6d6dbc6860401facadf1bc0365483b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:52.158425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1077/2154] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:51 2022] DELETE /identity/v3/users/9b6d6dbc6860401facadf1bc0365483b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:52.163343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.163558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.163859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.190264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:52.192777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:52.275613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.301351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1078/2155] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:52.303208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.305073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:52.391632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.437129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c5678-b9a6-4173-828a-a5872ebfd8e9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.442699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.443698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.443986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.444420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.455814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.457454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.473445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1078/2156] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:53.166226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.167834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:53.189140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15faa-4814-4eb1-920f-ce03a75c5648 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.219679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.220496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.221487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.221765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] PATH_INFO: `/v3/projects/7679a6dc178f4b12a6af2717cbfea33e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.225554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7679a6dc178f4b12a6af2717cbfea33e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.229375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.367172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7679a6dc178f4b12a6af2717cbfea33e', 'request_id': 'req-c7221270-40d3-447a-b7f9-be413f7a29e9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.378832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7679a6dc178f4b12a6af2717cbfea33e', 'request_id': 'req-c7221270-40d3-447a-b7f9-be413f7a29e9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.393218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7679a6dc178f4b12a6af2717cbfea33e', 'request_id': 'req-c7221270-40d3-447a-b7f9-be413f7a29e9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.421415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] The token cache is being invalidate because project 7679a6dc178f4b12a6af2717cbfea33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:53.423196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.425122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.430473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1079/2157] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:53 2022] DELETE /identity/v3/projects/7679a6dc178f4b12a6af2717cbfea33e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:53.435542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.450173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d56dc-8d8b-4935-9a3f-ca80d48527a0 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.568150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.569050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.569401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.569681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] PATH_INFO: `/v3/users/5c10381e66ee4d929a140ae02b1051ad` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.572014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c10381e66ee4d929a140ae02b1051ad)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.574008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.668500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c10381e66ee4d929a140ae02b1051ad'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.685656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c10381e66ee4d929a140ae02b1051ad'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.686862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c10381e66ee4d929a140ae02b1051ad'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.703035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c10381e66ee4d929a140ae02b1051ad'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.704098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c10381e66ee4d929a140ae02b1051ad'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:53.734814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1079/2158] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:53 2022] DELETE /identity/v3/users/5c10381e66ee4d929a140ae02b1051ad => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:54.539153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.540728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:54.564524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221270-40d3-447a-b7f9-be413f7a29e9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.656668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.658065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:54.690170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.690942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.692143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.692457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] PATH_INFO: `/v3/users/825af561e30446589faeb0d3afa23e52` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.694149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=825af561e30446589faeb0d3afa23e52)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.696159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.751643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:54.758085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c473-9c6c-4def-b56a-d5ff29011e39 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.765376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.767030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.767426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.767827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] PATH_INFO: `/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.769908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c34b709c40c4f98a4a6d15e6ce71627)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.771751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.784589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825af561e30446589faeb0d3afa23e52'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.786627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825af561e30446589faeb0d3afa23e52'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.803455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825af561e30446589faeb0d3afa23e52'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.804333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825af561e30446589faeb0d3afa23e52'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.816610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c4c54f03-ea60-434d-a799-9cea78031498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825af561e30446589faeb0d3afa23e52'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.847230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:54.863771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1080/2159] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:54 2022] DELETE /identity/v3/users/825af561e30446589faeb0d3afa23e52 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:54.865244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c34b709c40c4f98a4a6d15e6ce71627', 'request_id': 'req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.868147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a195-6859-4f5f-8b3f-25484bcfd48b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.868511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a195-6859-4f5f-8b3f-25484bcfd48b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.871147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a195-6859-4f5f-8b3f-25484bcfd48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.884671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c34b709c40c4f98a4a6d15e6ce71627', 'request_id': 'req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.903015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c34b709c40c4f98a4a6d15e6ce71627', 'request_id': 'req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.923363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fd43a195-6859-4f5f-8b3f-25484bcfd48b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:54.924744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fd43a195-6859-4f5f-8b3f-25484bcfd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:54.933906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] The token cache is being invalidate because project 7c34b709c40c4f98a4a6d15e6ce71627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:54.936986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.938632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-bf006e1d-fd9a-48d1-ab7e-34ddc84d3547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:54.946052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1080/2160] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:54 2022] DELETE /identity/v3/projects/7c34b709c40c4f98a4a6d15e6ce71627 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:54.949545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.951413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.951857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.010401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:55.013513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:55.031802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1081/2161] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:55.034615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1081/2162] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:55.044631 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.045193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.045377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.081063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.087604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:55.112210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:55.115825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:55.136308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1082/2163] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:55.146877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.163142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b231c1c-ff74-4897-81bc-50332b8ceedc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.176565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.176922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.177203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.177474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.178186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:55.178503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.180092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.215064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1083/2164] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:55.219678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.246578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263bc45-0e83-4fc2-a758-912c81fc657f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.247468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60a52e-b34b-49ed-b042-08fa01f321cd admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.259871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.260692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.260979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.261092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.261347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.261603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.261883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.262151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.262786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.267387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.303825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.308729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.333844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1082/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:30:55.341988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1084/2166] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:55.346594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.364472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39589c-0c96-4ab5-8d10-2eee4cfa11b5 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.545080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.545705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.546022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.546259 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.547121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.548887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.596535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.598447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1083/2167] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:55 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:55.611799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1feb4c-a910-42c8-a3da-4be6a8ab333d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.624744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.626786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.628068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.628666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.630446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.633107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.666076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.679421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1085/2168] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:55.683793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c86c6c-1c6d-4dd6-9a92-302a09bcbe78 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.689846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.690493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.690770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.691768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.692515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.694125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5881df5-5b80-4c55-94ad-711b91d5911c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.700584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.719663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.740122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1084/2169] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:30:55.754716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ec2a-d5e3-4e79-8c47-4f377eaf23ac admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.761058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.762646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.762932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.763279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] PATH_INFO: `/v3/projects/36a81e9ac70748cf9574a9c089390fd5/users/fe9633e6f4fa46139b2896d6491ac46b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.790678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a81e9ac70748cf9574a9c089390fd5, user_id=fe9633e6f4fa46139b2896d6491ac46b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.793637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.812762 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1086/2170] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:55 2022] PUT /identity/v3/projects/36a81e9ac70748cf9574a9c089390fd5/users/fe9633e6f4fa46139b2896d6491ac46b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:55.826160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.826716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.827067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.893589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] MFA Rules not processed for user `fe9633e6f4fa46139b2896d6491ac46b`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:55.895348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:56.076577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1085/2171] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:56.089057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.232599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3e40-71be-4033-8efb-b9b841a31e5b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.238890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.239940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.239940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.240186 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.241015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.243016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.281080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1087/2172] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:56 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:56.289939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.321743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e602b-7d90-4f56-a5f9-96fa537fda5e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.430554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.431170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.431449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.431717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.432437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.435163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.477753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.486279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:56.507812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:56.516909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090a4a-3613-4d9a-87ad-7cbe35b1ed89 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.530624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1086/2173] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:56 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:56.547270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.550096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.551356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.552057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] PATH_INFO: `/v3/projects/ea594a5db4d54a589f3d308aa8cdb4e7` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.553513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.555428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea594a5db4d54a589f3d308aa8cdb4e7)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.567102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.580696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da9a29-f83e-4f8f-98fb-04fe8bab2b42 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.595412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.596690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.596967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.597267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.600832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.602455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.639912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1087/2174] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:56.646595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.674154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576bf7a-31e1-484b-a2b9-78d637b8b4e8 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.700639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea594a5db4d54a589f3d308aa8cdb4e7', 'request_id': 'req-00ff69a2-4814-40c6-ad4a-058589ef3273'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:56.726893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea594a5db4d54a589f3d308aa8cdb4e7', 'request_id': 'req-00ff69a2-4814-40c6-ad4a-058589ef3273'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:56.738853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea594a5db4d54a589f3d308aa8cdb4e7', 'request_id': 'req-00ff69a2-4814-40c6-ad4a-058589ef3273'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:56.771564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] The token cache is being invalidate because project ea594a5db4d54a589f3d308aa8cdb4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:56.771876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:56.775837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:56.779258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1088/2175] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:56 2022] DELETE /identity/v3/projects/ea594a5db4d54a589f3d308aa8cdb4e7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:56.783670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.789426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:56.868469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.873268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.874552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.874828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] PATH_INFO: `/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44/users/b6bdeb003aa448b48aa568da78b8635f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.907462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:56.912205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00f321b7a2e492ab456f72b3f5f8e44, user_id=b6bdeb003aa448b48aa568da78b8635f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.916115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.946622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1088/2176] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:56 2022] PUT /identity/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44/users/b6bdeb003aa448b48aa568da78b8635f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:56.947784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.950198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:56.964865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff69a2-4814-40c6-ad4a-058589ef3273 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.091527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d74af3-2326-4c36-9d78-ad24efbbbde8 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.098345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.099105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.099385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.099656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.101388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.108299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.108733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.108733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.108733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.108733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] PATH_INFO: `/v3/users/c8800ca80f4848869d1954d2ac865453` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.110524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8800ca80f4848869d1954d2ac865453)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.113606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.129047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1089/2177] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:57.133266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.145033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb3525-b664-4738-98ba-ca3e41a1a79a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.165700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.166507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.167069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.167195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] PATH_INFO: `/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44/users/b6bdeb003aa448b48aa568da78b8635f/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.169975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00f321b7a2e492ab456f72b3f5f8e44, user_id=b6bdeb003aa448b48aa568da78b8635f, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.173456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630a5aa-0ddd-4224-9f7b-64c1ed06e664 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.186815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.196313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1090/2178] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:57 2022] PUT /identity/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44/users/b6bdeb003aa448b48aa568da78b8635f/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:30:57.217859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.221613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8800ca80f4848869d1954d2ac865453'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:57.222843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8800ca80f4848869d1954d2ac865453'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:57.223154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.223606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.264366 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8800ca80f4848869d1954d2ac865453'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:57.265782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8800ca80f4848869d1954d2ac865453'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:57.284592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6593fa38-1c80-4dbe-bb05-1d1f93ffdbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8800ca80f4848869d1954d2ac865453'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:57.334673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] MFA Rules not processed for user `b6bdeb003aa448b48aa568da78b8635f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:57.345853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1089/2179] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:30:56 2022] DELETE /identity/v3/users/c8800ca80f4848869d1954d2ac865453 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:57.348104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.348561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.348971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.353418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.415389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] MFA Rules not processed for user `30ccb163613a40ac8042f229199717f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:57.416550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.442969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1091/2180] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:57.460953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.465935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.545300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1090/2181] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:57.549860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.552280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.568102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddce63-0f5d-444d-85f9-e4ac54961d47 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.580312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.580902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.582199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.582886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.588308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.615849 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.675959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de28fe8-ace0-4a5f-b162-50761bcbae5a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.701052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.701831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.702213 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.702584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.713470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.715557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.716987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.719101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec83ed-3a70-47a6-b95a-bbcaa5be7fae service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.728031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1091/2182] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:57.731507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1092/2183] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:57.958351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.961642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:58.044544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:58.166048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2951e72-44ec-42c3-b370-081ccdcdaa4d service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.181414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.182430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.182717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.182994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] PATH_INFO: `/v3/projects/065f137ab27c4e64892c75ee6df5a5a3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.185824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065f137ab27c4e64892c75ee6df5a5a3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.187742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.276660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065f137ab27c4e64892c75ee6df5a5a3', 'request_id': 'req-a8f104e0-db48-47d6-9fa7-addc42701dd4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:58.287761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065f137ab27c4e64892c75ee6df5a5a3', 'request_id': 'req-a8f104e0-db48-47d6-9fa7-addc42701dd4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:58.312129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065f137ab27c4e64892c75ee6df5a5a3', 'request_id': 'req-a8f104e0-db48-47d6-9fa7-addc42701dd4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:58.327456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] The token cache is being invalidate because project 065f137ab27c4e64892c75ee6df5a5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:30:58.327766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:58.328530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a8f104e0-db48-47d6-9fa7-addc42701dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:30:58.330087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1092/2184] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:30:57 2022] DELETE /identity/v3/projects/065f137ab27c4e64892c75ee6df5a5a3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:58.336447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.336794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.337096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.375787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:58.380901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:58.489037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1093/2185] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:58.491328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.491494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.491774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.536427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:58.537673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:58.551948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1093/2186] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:58.558747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.578956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b85a3d-671c-4b20-8ece-d371808b0ee9 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.596118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.596871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.597759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.598145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.599335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:58.599335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.601896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.617865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1094/2187] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:58.622470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.639608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9705-f7ab-47f5-bf58-2b55f5873191 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.646344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c702a-5484-4cac-a886-30576475251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.647482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c702a-5484-4cac-a886-30576475251e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.647719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c702a-5484-4cac-a886-30576475251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.648043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c702a-5484-4cac-a886-30576475251e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.648793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c702a-5484-4cac-a886-30576475251e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.651103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c702a-5484-4cac-a886-30576475251e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.693768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1094/2188] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:30:58 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:58.697762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.711925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e785d9-9b51-4cc6-8c8b-1ca7146382ac admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.804851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.805567 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.805735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.806158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.806815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.808749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.834895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1095/2189] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:58 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:58.838843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c702a-5484-4cac-a886-30576475251e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.853595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c702a-5484-4cac-a886-30576475251e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.859915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.860643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.861000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.861350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.862070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.864109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.878369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1095/2190] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:58.879956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.890736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e02bc-e73e-4928-8a8f-32c233978aa5 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.901560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.902175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.902543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.902821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.903474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.905110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74ba4e-b1de-4335-bca4-759992dfd617 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.929962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1096/2191] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:30:58.933809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.950524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c2730-ac3a-4884-9340-30f904845940 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.955994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.957468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.957806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.958082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] PATH_INFO: `/v3/projects/5e4df6feaf3c4417ad1cdf3f4e9a2594/users/15b9a641910f4626a2714fbb66257974/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.976829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4df6feaf3c4417ad1cdf3f4e9a2594, user_id=15b9a641910f4626a2714fbb66257974, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.979526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.993658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1096/2192] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:30:58 2022] PUT /identity/v3/projects/5e4df6feaf3c4417ad1cdf3f4e9a2594/users/15b9a641910f4626a2714fbb66257974/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:59.005568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f9ab-1d45-4db1-a8e8-0ccc91179d2e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.005867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f9ab-1d45-4db1-a8e8-0ccc91179d2e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.006582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f9ab-1d45-4db1-a8e8-0ccc91179d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.037515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9582f9ab-1d45-4db1-a8e8-0ccc91179d2e None None] MFA Rules not processed for user `15b9a641910f4626a2714fbb66257974`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:59.038828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9582f9ab-1d45-4db1-a8e8-0ccc91179d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:59.048478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:59.114121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1097/2193] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:30:59.125022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:59.126838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:59.198392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1297de1-9b44-4a9d-844f-b2c7c951f493 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:59.202034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.202645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.202918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.203317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.212686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.214135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.217835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:59.234946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1097/2194] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:30:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:30:59.945976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beff73-9bee-4809-9892-35ac0a955829 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.947420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beff73-9bee-4809-9892-35ac0a955829 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.948039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beff73-9bee-4809-9892-35ac0a955829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:00.002802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-54beff73-9bee-4809-9892-35ac0a955829 None None] MFA Rules not processed for user `30ccb163613a40ac8042f229199717f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:00.004700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-54beff73-9bee-4809-9892-35ac0a955829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:00.093120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1098/2195] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:30:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:00.616458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:00.618101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:00.704109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f3157-5910-4d28-a256-77a29c9bea45 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:00.723331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:00.723331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:00.723331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:00.723331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] PATH_INFO: `/v3/users/fe9633e6f4fa46139b2896d6491ac46b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:00.723704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9633e6f4fa46139b2896d6491ac46b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:00.723704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:00.799823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9633e6f4fa46139b2896d6491ac46b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:00.800769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9633e6f4fa46139b2896d6491ac46b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:00.814608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9633e6f4fa46139b2896d6491ac46b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:00.815296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9633e6f4fa46139b2896d6491ac46b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:00.826293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9633e6f4fa46139b2896d6491ac46b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:00.867143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1098/2196] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:31:00 2022] DELETE /identity/v3/users/fe9633e6f4fa46139b2896d6491ac46b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:00.868734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184203-746b-4a38-adb4-f333192e807b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:00.869231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184203-746b-4a38-adb4-f333192e807b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:00.869546 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184203-746b-4a38-adb4-f333192e807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:00.921492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c0184203-746b-4a38-adb4-f333192e807b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:00.926132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c0184203-746b-4a38-adb4-f333192e807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:01.063828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1099/2197] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:31:01.075472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.085165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:01.192189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c56eb-ba3f-4113-9375-a82af67e1eaa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.198886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.199531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.199844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.200657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.224803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.243525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.258122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1099/2198] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:31:01.670548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184203-746b-4a38-adb4-f333192e807b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.672072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c0184203-746b-4a38-adb4-f333192e807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:01.685879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184203-746b-4a38-adb4-f333192e807b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.691580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.692619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.693029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.693366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] PATH_INFO: `/v3/projects/36a81e9ac70748cf9574a9c089390fd5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.694693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a81e9ac70748cf9574a9c089390fd5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.696773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.750232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a81e9ac70748cf9574a9c089390fd5', 'request_id': 'req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.751472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:01.759344 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a81e9ac70748cf9574a9c089390fd5', 'request_id': 'req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.767475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a81e9ac70748cf9574a9c089390fd5', 'request_id': 'req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.779263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] The token cache is being invalidate because project 36a81e9ac70748cf9574a9c089390fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:31:01.779551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.780212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.781970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1100/2199] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:31:01 2022] DELETE /identity/v3/projects/36a81e9ac70748cf9574a9c089390fd5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:01.785993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.796545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1433fa-2dbf-464a-9a2c-57f44da4137a service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.818097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:01.889725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.890398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.890681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.890961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] PATH_INFO: `/v3/users/b6bdeb003aa448b48aa568da78b8635f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.892118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bdeb003aa448b48aa568da78b8635f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.894227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.956980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bdeb003aa448b48aa568da78b8635f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.957663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bdeb003aa448b48aa568da78b8635f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.965552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bdeb003aa448b48aa568da78b8635f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.966348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bdeb003aa448b48aa568da78b8635f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:01.982570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bdeb003aa448b48aa568da78b8635f'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.013787 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1100/2200] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:31:01 2022] DELETE /identity/v3/users/b6bdeb003aa448b48aa568da78b8635f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:02.427392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.437464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:02.453539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0cdf-976e-4b24-98d5-15c9fd7a9549 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.516153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.516824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.517106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.517384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] PATH_INFO: `/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.518520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00f321b7a2e492ab456f72b3f5f8e44)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.520666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.569295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00f321b7a2e492ab456f72b3f5f8e44', 'request_id': 'req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.576004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00f321b7a2e492ab456f72b3f5f8e44', 'request_id': 'req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.586101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00f321b7a2e492ab456f72b3f5f8e44', 'request_id': 'req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.600742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] The token cache is being invalidate because project f00f321b7a2e492ab456f72b3f5f8e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:31:02.601037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.601695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-80ffb20d-7f9b-4c1e-996d-a65a940fc4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:02.603338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1101/2201] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:31:02 2022] DELETE /identity/v3/projects/f00f321b7a2e492ab456f72b3f5f8e44 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:09.145700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.147142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.160023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f068-536e-4261-8c25-4c4819882954 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.219558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.220197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.220564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.220845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] PATH_INFO: `/v3/users/15b9a641910f4626a2714fbb66257974` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.221972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b9a641910f4626a2714fbb66257974)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.224517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.267905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:09.274141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b9a641910f4626a2714fbb66257974'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.274826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b9a641910f4626a2714fbb66257974'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.282344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b9a641910f4626a2714fbb66257974'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.283043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b9a641910f4626a2714fbb66257974'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.291353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b9a641910f4626a2714fbb66257974'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.310495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1101/2202] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:31:09 2022] DELETE /identity/v3/users/15b9a641910f4626a2714fbb66257974 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:09.314940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.315418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.315510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.335557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:09.336831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.395892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1102/2203] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:09.405560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.406944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.464758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d6bcc-f715-45bd-9aca-472ebf63674e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.468735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.469442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.469723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.469991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.479890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.481347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34c9ad-6436-4181-afbf-cc4d860527a5 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.491765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1102/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:31:09.843185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.844756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.870080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48be97-cc77-41fd-be72-547623136d2c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.875696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.876262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.876550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.876846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] PATH_INFO: `/v3/projects/5e4df6feaf3c4417ad1cdf3f4e9a2594` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.877915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4df6feaf3c4417ad1cdf3f4e9a2594)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.880011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.882647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:09.930872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4df6feaf3c4417ad1cdf3f4e9a2594', 'request_id': 'req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.937554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4df6feaf3c4417ad1cdf3f4e9a2594', 'request_id': 'req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.944703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4df6feaf3c4417ad1cdf3f4e9a2594', 'request_id': 'req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.955285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] The token cache is being invalidate because project 5e4df6feaf3c4417ad1cdf3f4e9a2594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:31:09.955571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.956389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1c7c7ae4-a0ac-4888-8123-8f7076a549bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:09.958074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1103/2205] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:31:09 2022] DELETE /identity/v3/projects/5e4df6feaf3c4417ad1cdf3f4e9a2594 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:09.963749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.964032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.964350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.984674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:09.985883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:10.046517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1103/2206] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:10.055402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.055682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.055951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.077496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:10.078819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:10.095654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1104/2207] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:10.100537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.116953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e7c08-873d-4a7c-9a04-d6c4fadcaf37 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.137118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.137758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.138066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.138336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.138998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:10.139294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.140918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.153434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1104/2208] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:10.158114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.168237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7656bbf-d03a-4748-a672-acc3147f2eb1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.173052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.173781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.173912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.174176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.174884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.176790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.205329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1105/2209] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:31:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:10.209841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.220074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c6807-6206-432e-9a3b-43aa1c6edc76 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.268726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.282562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.283166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.283443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.283713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.284387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.286868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.307612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1105/2210] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:10 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:10.311720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.322211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0674b-50a0-447d-9aa1-c0e2f04f4c56 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.327014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.327619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.327914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.328188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.328861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.330438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.341289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1106/2211] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:31:10.345251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.356513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756505-54e0-47d5-9399-ed8bc7fc973d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.362368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.363394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.363576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.364062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.364886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.366469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8549495-1362-491c-b956-3745958e04f0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.377638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1106/2212] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:31:10.382144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.393167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70bcc1-21e0-4ded-83c1-dc2ca86c247e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.398718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.399459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.399744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.400018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] PATH_INFO: `/v3/projects/62cb730ec8d4449cafadd891a33df9e2/users/2e229e28082641d992a36978597a2b01/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.417234 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cb730ec8d4449cafadd891a33df9e2, user_id=2e229e28082641d992a36978597a2b01, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.419911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.435318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1107/2213] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:31:10 2022] PUT /identity/v3/projects/62cb730ec8d4449cafadd891a33df9e2/users/2e229e28082641d992a36978597a2b01/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:10.446206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fdf3-b076-425b-b466-2a6c1539c698 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.446513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fdf3-b076-425b-b466-2a6c1539c698 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.446904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8fdf3-b076-425b-b466-2a6c1539c698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.478578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-72e8fdf3-b076-425b-b466-2a6c1539c698 None None] MFA Rules not processed for user `2e229e28082641d992a36978597a2b01`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:10.479807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-72e8fdf3-b076-425b-b466-2a6c1539c698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:10.548201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1107/2214] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:10.558919 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.560663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:10.620381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658388e5-ecdc-409c-abb5-72440a438550 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.623489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.624084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.624410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.624682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.633452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.634868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.644025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1108/2215] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:31:19.284518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5845502-e180-4a89-806e-654be36ecd44 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.285959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5845502-e180-4a89-806e-654be36ecd44 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.285959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5845502-e180-4a89-806e-654be36ecd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.320153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c5845502-e180-4a89-806e-654be36ecd44 None None] MFA Rules not processed for user `48e588ff6d5a4a37b902abadfb6d04fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:19.321559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c5845502-e180-4a89-806e-654be36ecd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:19.391254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1108/2216] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:20.639762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:21.551544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:21.636181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3591-c75f-4d8d-be05-b75834f87324 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:21.642551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:21.643253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.643555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.643897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] PATH_INFO: `/v3/users/2e229e28082641d992a36978597a2b01` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.645916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e229e28082641d992a36978597a2b01)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:21.647988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:21.707439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e229e28082641d992a36978597a2b01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:21.718605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e229e28082641d992a36978597a2b01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:21.719586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e229e28082641d992a36978597a2b01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:21.730239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e229e28082641d992a36978597a2b01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:21.731387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e229e28082641d992a36978597a2b01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:21.756528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1109/2217] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:31:21 2022] DELETE /identity/v3/users/2e229e28082641d992a36978597a2b01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:21.758174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.758622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.758712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.780725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:21.782091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:21.865071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1109/2218] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:21.876479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:21.877889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:21.945636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8434f-0c97-43bb-b218-05d8a5b9ab9c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:21.951119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:21.951730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.952040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.952348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.962817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:21.964226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd6dbe-62f7-40c4-a900-ca15c7e3f883 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:21.974337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1110/2219] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:31:22.304633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:22.306779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:22.319424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:22.323570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533815fb-a80f-451e-a79b-1cd269cc907f None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:22.329780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:22.330640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:22.331068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:22.332037 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] PATH_INFO: `/v3/projects/62cb730ec8d4449cafadd891a33df9e2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:22.334511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62cb730ec8d4449cafadd891a33df9e2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:22.338815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:22.423083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62cb730ec8d4449cafadd891a33df9e2', 'request_id': 'req-dcb830e4-4280-46cf-9156-bac8b7408638'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:22.431941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62cb730ec8d4449cafadd891a33df9e2', 'request_id': 'req-dcb830e4-4280-46cf-9156-bac8b7408638'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:22.444841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62cb730ec8d4449cafadd891a33df9e2', 'request_id': 'req-dcb830e4-4280-46cf-9156-bac8b7408638'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:22.459576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] The token cache is being invalidate because project 62cb730ec8d4449cafadd891a33df9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:31:22.459868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:22.460690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:31:22.462690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1110/2220] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:31:22 2022] DELETE /identity/v3/projects/62cb730ec8d4449cafadd891a33df9e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:31.970167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:32.453946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:37.523065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fe73-d525-43f6-9309-da99aa366cce None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:37.523606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fe73-d525-43f6-9309-da99aa366cce None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:37.523606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174fe73-d525-43f6-9309-da99aa366cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:37.543124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8174fe73-d525-43f6-9309-da99aa366cce None None] MFA Rules not processed for user `683c4d41fa4a452ba8210e62dd43f7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:37.544339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8174fe73-d525-43f6-9309-da99aa366cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:37.603720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1111/2221] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:31:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:31:44.800383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:44.803313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:44.907573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.908913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.909400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.931342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb830e4-4280-46cf-9156-bac8b7408638 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:44.935210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:44.936871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:44.947102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:44.947755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.948033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.948338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.950358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:45.041613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1112/2222] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:31:45.042241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.043915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6687d-b7a3-4a94-abc1-57912b72e3eb service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.056948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1111/2223] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:31:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:31:45.093797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.099885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:45.207235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b51c4-c78e-4334-b843-cba4b70f10a4 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.214954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.215837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.216276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.216684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.227964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.229939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f65a5-622e-4759-b3db-7c9f88da1cdb service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.256458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1113/2224] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:31:55.051561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=50 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:55.237927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=82 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:08.349671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.350248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.350248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.351603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1112/2225] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:08.357704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3356c-9138-41cf-a77a-29c392621d0c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.358249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3356c-9138-41cf-a77a-29c392621d0c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.358458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3356c-9138-41cf-a77a-29c392621d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.380986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-98f3356c-9138-41cf-a77a-29c392621d0c None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:08.382215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-98f3356c-9138-41cf-a77a-29c392621d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:08.453709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1114/2226] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:32:08.466629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:08.468454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:08.550581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb50589-8aa8-4b8d-bbd2-e8e5b965cba4 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:08.554148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:08.554749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.555028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.555299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.565251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:08.567141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2294a-0b0b-49de-8fdb-8686064821d8 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:08.577692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1113/2227] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:32:08.648425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.648633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.648907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.650270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1115/2228] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:08.655203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056fe10-b77d-4be1-8c09-3559e71c5ed0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.655487 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056fe10-b77d-4be1-8c09-3559e71c5ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.655763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056fe10-b77d-4be1-8c09-3559e71c5ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.677097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9056fe10-b77d-4be1-8c09-3559e71c5ed0 None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:08.678304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9056fe10-b77d-4be1-8c09-3559e71c5ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:08.684862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1114/2229] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:32:08.696542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:08.698223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:08.711590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be65b2-a242-4302-9a65-4851f458c0a3 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:08.715074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:08.715654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.715927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.716195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.723682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:08.725104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293e87-e2fd-4c76-965a-fc59893df1ed service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:08.733439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1116/2230] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:32:08.817595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728b54-e4b8-41a9-bbe2-cda05a145cc8 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.817966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728b54-e4b8-41a9-bbe2-cda05a145cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.818313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728b54-e4b8-41a9-bbe2-cda05a145cc8 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.819778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1115/2231] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:08.825291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.825642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.825977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.848684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:08.849892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:08.859311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1117/2232] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:32:08.870164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05728b54-e4b8-41a9-bbe2-cda05a145cc8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:08.881529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05728b54-e4b8-41a9-bbe2-cda05a145cc8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:08.885322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:08.885990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:08.886331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:08.886666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:08.897796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:08.899357 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f1f6a-c033-4b4f-8a20-d6c4406a3674 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:08.910864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1116/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:32:17.314692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.316350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:17.390209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c576-4796-4960-86d3-f89e6397521d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.395198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.395818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.396098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.396396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.398138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:17.411582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.413284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.424129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1118/2234] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:32:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:32:17.568147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23016-b7dd-4317-9d30-45a5b54b9acf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.568147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23016-b7dd-4317-9d30-45a5b54b9acf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.568147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb23016-b7dd-4317-9d30-45a5b54b9acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.592871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2fb23016-b7dd-4317-9d30-45a5b54b9acf None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:17.594540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2fb23016-b7dd-4317-9d30-45a5b54b9acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:17.603981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1117/2235] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:32:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:32:17.617568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.629942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de0d-d0b9-41bf-b3fb-9165d31733a9 service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.637264 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.637889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.638167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.638438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.662822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.664464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc961645-9620-43c6-900a-e27235e14dfe service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.674462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1119/2236] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:32:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:32:27.585793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=47 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:27.669945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=78 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:48.761524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:48.762022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:48.762120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:48.781099 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] MFA Rules not processed for user `1e391d2591734de3bbe2515c703c4602`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:48.782481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:48.855482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1118/2237] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:32:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:32:55.613446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff283c5-d675-4caf-944e-2c47ac9cfe0b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:55.613945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff283c5-d675-4caf-944e-2c47ac9cfe0b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:55.614045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff283c5-d675-4caf-944e-2c47ac9cfe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:55.635013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-aff283c5-d675-4caf-944e-2c47ac9cfe0b None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:55.636215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aff283c5-d675-4caf-944e-2c47ac9cfe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:55.643342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1120/2238] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:32:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:32:55.665535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:55.677903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30dee5-3ddb-43ad-9a5f-0ee6a585ff5d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:55.682800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:55.683414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:55.683690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:55.683964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:55.692467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:55.694252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:55.702546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1119/2239] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:32:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:33:05.630130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:05.699252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=31 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:30.976479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:30.976995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:30.976995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:30.996446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] MFA Rules not processed for user `48e588ff6d5a4a37b902abadfb6d04fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:33:30.997641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:33:31.051288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1121/2240] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:33:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:33:41.044664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=25 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:00.736201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:00.737677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:00.800655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155873eb-47cc-49bb-976f-49be6c042278 service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:00.805088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:00.806287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:00.806600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:00.806868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] PATH_INFO: `/v3/users/1e391d2591734de3bbe2515c703c4602` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:00.807980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e391d2591734de3bbe2515c703c4602)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:00.809951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:00.856255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e391d2591734de3bbe2515c703c4602'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:00.856916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e391d2591734de3bbe2515c703c4602'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:00.864149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e391d2591734de3bbe2515c703c4602'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:00.864779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e391d2591734de3bbe2515c703c4602'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:00.872158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2909ace5-548f-494b-bc3c-2273fe863140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e391d2591734de3bbe2515c703c4602'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:00.889302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1120/2241] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:34:00 2022] DELETE /identity/v3/users/1e391d2591734de3bbe2515c703c4602 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:34:01.056425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.067217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed833b5-1801-4869-a055-8a026ddcc15b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.125577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.126199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.126472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.126739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] PATH_INFO: `/v3/projects/241e5470186a4c93b2465ecc84d80603` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.127836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241e5470186a4c93b2465ecc84d80603)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.129756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.174932 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241e5470186a4c93b2465ecc84d80603', 'request_id': 'req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:01.181177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241e5470186a4c93b2465ecc84d80603', 'request_id': 'req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:01.188272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241e5470186a4c93b2465ecc84d80603', 'request_id': 'req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:01.198555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] The token cache is being invalidate because project 241e5470186a4c93b2465ecc84d80603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:34:01.198845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:01.199488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9db019a8-6f2d-4c4d-8a7a-47145750d8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:34:01.200993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1122/2242] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:34:01 2022] DELETE /identity/v3/projects/241e5470186a4c93b2465ecc84d80603 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:34:01.206164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.206475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.206751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.226940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:01.229106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:01.288089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1121/2243] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:01.297034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d083fd-a174-4457-8158-d5771883876d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.297319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d083fd-a174-4457-8158-d5771883876d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.297586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d083fd-a174-4457-8158-d5771883876d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.318482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f2d083fd-a174-4457-8158-d5771883876d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:01.319707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f2d083fd-a174-4457-8158-d5771883876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:01.328673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1123/2244] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:01.332930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.343173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315dcd9-0ef9-4696-bab4-0226520174db None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.347741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.348388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.349077 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.349666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.350363 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:01.350662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.353135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.369173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1122/2245] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:01.373145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d083fd-a174-4457-8158-d5771883876d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.383008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d083fd-a174-4457-8158-d5771883876d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.388070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.388770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.389105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.389454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.390146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.392084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.431429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1124/2246] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:34:01.435381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.445977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52591-bb1c-46bd-b3b4-7fd8ff353ba5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.456782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:01.525181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.525921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.526323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.526693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.527327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.529301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.551095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1123/2247] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:34:01.555305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.572437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894158b-b092-4edd-884b-7d2e7964cf32 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.577726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.578333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.578612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.578910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.579606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.581240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.592500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1125/2248] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:34:01.596501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.606862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da22-2f00-491d-b295-6c6ebf3ba286 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.611589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.612349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.612693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.613030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.613741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.615331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6d513-6717-42bf-aa51-9e1e02c0f778 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.628325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1124/2249] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:34:01.630406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.641389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877551-faa1-403b-bfca-2b953857d64e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.647238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.648142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.648586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.649061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] PATH_INFO: `/v3/projects/774c63ee5fc74c619f63c64590c154e4/users/57daecc890ac421b8eb7b5fb4bf25411/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.667279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=774c63ee5fc74c619f63c64590c154e4, user_id=57daecc890ac421b8eb7b5fb4bf25411, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.670723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.690573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1126/2250] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:34:01 2022] PUT /identity/v3/projects/774c63ee5fc74c619f63c64590c154e4/users/57daecc890ac421b8eb7b5fb4bf25411/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:34:01.697449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed98e-fefa-4227-b6ee-694fad678085 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.697914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed98e-fefa-4227-b6ee-694fad678085 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.698193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daed98e-fefa-4227-b6ee-694fad678085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.719414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5daed98e-fefa-4227-b6ee-694fad678085 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:01.720819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5daed98e-fefa-4227-b6ee-694fad678085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:01.781810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1125/2251] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:01.791666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.793421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:01.824381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:01.853750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b5c8-2430-46a2-9eb1-1a28ed9a8c4e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.857408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.858004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.858284 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.858572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.867337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:01.869411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:01.878599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1127/2252] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:06.117852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.118825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.119238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.150972 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] MFA Rules not processed for user `57daecc890ac421b8eb7b5fb4bf25411`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:06.152578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:06.222552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1126/2253] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:06.229916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.248362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451ef72-877c-4e0f-aa2a-4df6a1e41187 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.257482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.257820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.258669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.259016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.260672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.262474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.294224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1128/2254] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:34:06 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:34:06.296791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.310287 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f979b-3193-485b-97d8-49f1d9974c02 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.324546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:06.374911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.375536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.375818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.376090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.376784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.378710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.401525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1127/2255] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:34:06 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:34:06.405459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.416060 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00768aa-e7b4-4059-899d-2f528af9f884 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.420707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.421316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.421591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.421882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.422533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.424113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.436633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1129/2256] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:34:06.441069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.452620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928768cb-e55b-4462-b2fc-e7ee1ecd7b1e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.457365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.458031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.458300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.458572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] PATH_INFO: `/v3/projects/297020a5021c4784889de1ea9e461280/users/a9c5d43e80d742ff9f750b057dcd11a5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.475082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297020a5021c4784889de1ea9e461280, user_id=a9c5d43e80d742ff9f750b057dcd11a5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.477805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.496294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1128/2257] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:34:06 2022] PUT /identity/v3/projects/297020a5021c4784889de1ea9e461280/users/a9c5d43e80d742ff9f750b057dcd11a5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:34:06.500133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.511833 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdc888-2a70-4ff9-a34a-2ed3d683c358 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.572601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.573214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.573487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.573753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.574392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.576136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.590500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1130/2258] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:34:06.591721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.603825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfeca-60c9-49dc-a87c-70d1509a6838 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:06.608739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:06.609408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.609688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.609988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] PATH_INFO: `/v3/projects/297020a5021c4784889de1ea9e461280/users/a9c5d43e80d742ff9f750b057dcd11a5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.611913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297020a5021c4784889de1ea9e461280, user_id=a9c5d43e80d742ff9f750b057dcd11a5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:06.614613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:06.626625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1129/2259] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:34:06 2022] PUT /identity/v3/projects/297020a5021c4784889de1ea9e461280/users/a9c5d43e80d742ff9f750b057dcd11a5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:34:06.963211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:06.966956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:07.052659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a7ce-26b4-42fc-aa56-8dc558c40a5f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:07.122528 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:07.127788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:07.128713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:07.129126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:07.129474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:07.132149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:07.212499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:07.214247 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79c093-b798-41ff-a4ad-127ac6378cab service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:07.227901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1131/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:08.552409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.585736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4e606-d1bd-4b73-a738-56c29d1c4dc8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.604416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.605217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.607086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.607935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.613729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:08.643064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.645591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.693870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1130/2261] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:12.365837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcd5c-8c71-4b4a-ad0a-403cb8cee125 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.366314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcd5c-8c71-4b4a-ad0a-403cb8cee125 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.366423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcd5c-8c71-4b4a-ad0a-403cb8cee125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.432297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-86adcd5c-8c71-4b4a-ad0a-403cb8cee125 None None] MFA Rules not processed for user `a9c5d43e80d742ff9f750b057dcd11a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:12.433517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-86adcd5c-8c71-4b4a-ad0a-403cb8cee125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:12.532002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1132/2262] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:12.548349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.550298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:12.568545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089418-f6c6-4b1f-af39-b05591184283 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.580435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.581318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.581807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.582410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.700041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:12.703537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3af1-b591-4d70-9157-9264718e8dd9 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:12.723623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1131/2263] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:13.357803 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:13.358092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:13.358383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:13.383118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] MFA Rules not processed for user `a8f59b47841a4b62b8e368562c7bd210`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:13.384408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:13.450377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1133/2264] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:17.092539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:17.093092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:17.093092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:17.118619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] MFA Rules not processed for user `683c4d41fa4a452ba8210e62dd43f7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:17.120004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:17.185963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:17.202537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1132/2265] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:20.017203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:20.018645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:20.035410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c607b8-27ad-4ae0-97f8-0b06c9304cea None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:20.040947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:20.041562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:20.041836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:20.042128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:20.043817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:20.055542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:20.057124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:20.065993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1134/2266] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:22.854925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:22.869300 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40e3f7-3677-4748-9f7c-661ae9aa4656 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:22.874968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:22.875608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:22.875953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:22.876326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:22.878314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:22.890804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:22.892455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309c59-dee3-4ad2-a9e7-dc4838eb7471 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:22.901961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1133/2267] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:30.061870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:31.751108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:31.760949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864e010-13c3-4cee-8c7c-587df493a56a service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:31.764942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:31.765625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:31.765973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:31.766251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:31.768096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:31.778683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:31.780281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:31.788497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1135/2268] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:32.897583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=7 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:34.252977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:34.253579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:34.253579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:34.272294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] MFA Rules not processed for user `57daecc890ac421b8eb7b5fb4bf25411`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:34.274257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:34.281008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1134/2269] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:34:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:34:34.593914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:34.603175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850598dd-a082-4a3d-9bbd-66a8943076f2 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:34.606737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:34.607328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:34.607664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:34.607863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:34.609587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:34.620066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:34.621508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:34.630019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1136/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:35.092744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:35.097270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:35.179244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be184da-8764-4a90-aa3b-2bff462146dc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:35.184243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:35.184867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:35.185143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:35.185410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:35.187106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:35.199138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:35.200772 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:35.211242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1135/2271] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:43.458418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:43.472597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa1e62-bb22-4726-8d48-2540fcc15344 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:43.478991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:43.479696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:43.480049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:43.480424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:43.482201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:43.542410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:43.544001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:43.553122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1137/2272] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:45.206319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=35 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:53.166556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:53.175798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a475ca-1be0-47e1-8924-755667f9500e service placement] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:53.180277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:53.180868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:53.181144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:53.181414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:53.190177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:53.191597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:53.201421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1136/2273] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:34:53.548794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=40 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:55.115503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:55.127101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd997-6728-478a-8f64-788e02617ae4 service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:55.133068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:55.133689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:55.134020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:55.134300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:55.136161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:55.149689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:55.151340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30ffb9-dc32-422b-b643-2dc4635ad1a9 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:55.163104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1138/2274] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:34:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:01.306578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:01.318683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6d86d-fb97-443c-9147-7714a80d6fdb service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:01.324066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:01.324685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.324955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.325224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.326942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:01.340056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:01.341664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12ab5f-09ee-43f1-be59-10c00d2c8683 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:01.355967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1137/2275] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:01.701572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10a81-b683-4d93-9ecd-c2c0a4e7186a None None] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.702143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10a81-b683-4d93-9ecd-c2c0a4e7186a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.702256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10a81-b683-4d93-9ecd-c2c0a4e7186a None None] PATH_INFO: `/` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.703817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1139/2276] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:01.711307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6f11-3ae3-4bbf-8e64-a8242e6d5326 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.711906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6f11-3ae3-4bbf-8e64-a8242e6d5326 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.712239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6f11-3ae3-4bbf-8e64-a8242e6d5326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.735454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b0bc6f11-3ae3-4bbf-8e64-a8242e6d5326 None None] MFA Rules not processed for user `a938ac6d7d3b42ab9e8aa8b2b9ea9410`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:01.737156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0bc6f11-3ae3-4bbf-8e64-a8242e6d5326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:01.744900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1138/2277] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:35:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:01.760208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10a81-b683-4d93-9ecd-c2c0a4e7186a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:01.778091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10a81-b683-4d93-9ecd-c2c0a4e7186a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:01.783414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:01.784778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.786249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.786607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.799503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:01.801228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247267d1-ddf3-4d0c-aa81-e0c343dc4ed4 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:01.812229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1140/2278] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:01.919549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf132c8-bae3-4b86-ac60-dae882c96f74 None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.919829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf132c8-bae3-4b86-ac60-dae882c96f74 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.920097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf132c8-bae3-4b86-ac60-dae882c96f74 None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.921453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1139/2279] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 05:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:01.926595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:01.926898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:01.927175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:01.951770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] MFA Rules not processed for user `2eadef8d5ac94a3eb3da8d0d3a66d7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:01.953582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:01.966737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1141/2280] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 05:35:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:01.983306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf132c8-bae3-4b86-ac60-dae882c96f74 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:01.998325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf132c8-bae3-4b86-ac60-dae882c96f74 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:02.006481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:02.007100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:02.007553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:02.007974 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:02.053639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:02.055408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69410c-511b-4ed6-bc7c-7346fd822698 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:02.102044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1140/2281] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:02.149076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:02.161530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ae2fd-f6b0-44e4-bd91-0f288f3a2457 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:02.171551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:02.172179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:02.172493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:02.172762 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:02.174570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:02.189692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:02.191419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:02.208142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1142/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:05.110110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e30b-d64d-49db-a68b-ba3ab431cb26 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:05.110579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e30b-d64d-49db-a68b-ba3ab431cb26 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:05.110579 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e30b-d64d-49db-a68b-ba3ab431cb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:05.129479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-bf27e30b-d64d-49db-a68b-ba3ab431cb26 None None] MFA Rules not processed for user `48e588ff6d5a4a37b902abadfb6d04fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:05.131276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bf27e30b-d64d-49db-a68b-ba3ab431cb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:05.138039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1141/2283] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:12.199909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=21 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:15.124610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=24 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:37.702749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:37.712771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45dad-053b-4d3f-bb9a-7e5f1b0b94d6 service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:37.716326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:37.716914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:37.717191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:37.717461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:37.719136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:37.781153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:37.782647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19c35c-48ee-4316-b1f5-20e17ad772b8 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:37.792156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1143/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:41.165685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.166191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.166303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] PATH_INFO: `/` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.167543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1142/2285] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 05:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:41.172503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.172963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.173074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.193200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] MFA Rules not processed for user `35bda10601d04f22a509cfb45e19c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:41.194404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.202956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1144/2286] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 05:35:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:41.220197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.222430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.222615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.233679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00f3cf-8e1b-404d-a188-61cf7c63e212 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.239438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.240121 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.240630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.240901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.242211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c993656-fda1-4a32-aad6-6782545a6acd None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.242630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.246922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.247541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.247817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.248089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] PATH_INFO: `/v3/users/683c4d41fa4a452ba8210e62dd43f7f4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.249275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=683c4d41fa4a452ba8210e62dd43f7f4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.251217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.254288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.256108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.266071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1145/2287] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:35:41.299598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683c4d41fa4a452ba8210e62dd43f7f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.300291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683c4d41fa4a452ba8210e62dd43f7f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.308041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683c4d41fa4a452ba8210e62dd43f7f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.308660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683c4d41fa4a452ba8210e62dd43f7f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.315440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ace569a-a362-4391-b12c-790e61e0ba3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683c4d41fa4a452ba8210e62dd43f7f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.334231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1143/2288] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:41 2022] DELETE /identity/v3/users/683c4d41fa4a452ba8210e62dd43f7f4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:41.507709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.518361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547d15-43e7-43b2-9ef2-10b971f784ab service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.579314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.579937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.580217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.580531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] PATH_INFO: `/v3/projects/01c8f7766cb646b7b646a53c9de8e91a` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.581640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c8f7766cb646b7b646a53c9de8e91a)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.584053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.629924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c8f7766cb646b7b646a53c9de8e91a', 'request_id': 'req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.636073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c8f7766cb646b7b646a53c9de8e91a', 'request_id': 'req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.642998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c8f7766cb646b7b646a53c9de8e91a', 'request_id': 'req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.654549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] The token cache is being invalidate because project 01c8f7766cb646b7b646a53c9de8e91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:41.654826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.655461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-0f3ff91f-2e2a-4cc6-9092-dc21c1edb403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:41.656981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1146/2289] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:41 2022] DELETE /identity/v3/projects/01c8f7766cb646b7b646a53c9de8e91a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:41.662708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.662996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.663274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.682739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:41.683991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.745409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1144/2290] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:41.753088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.754820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.755108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.763718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:41.777640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:41.779130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.787700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1147/2291] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:41.791998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.802472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e0f5f-0ee2-4990-81bd-6815b96ca9a7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.807149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.807781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.808051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.808337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.808973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:35:41.809269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.810855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.822550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1145/2292] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:35:41.827081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.837027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5392-fcdc-44f7-a0ed-c8fd0cfa91f9 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.842039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.842642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.842929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.843185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.843811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.845702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.877230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1148/2293] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:41 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:35:41.878606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.888971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144316a-387f-47f8-8119-b705954f04a2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.945458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.946063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.946340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.946632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.947262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.949235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.957978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:41.972309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1146/2294] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:41 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:41.976925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:41.987391 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a164-4743-4db9-98cc-b75ef1be48f6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.992080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.992687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.992961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.993227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.993853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.995413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.006130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1149/2295] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:35:42.010028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.021736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff86dac-ade2-479d-a3df-3295bce7370d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.027297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.028321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.028809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.029297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.030460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.033526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb46e3-3b73-4eea-8b5b-34c0f0f58bd2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.046147 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1147/2296] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:42.050388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.060231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cc3b7-a816-4ad7-bf0b-1122e6a5b540 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.064815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.065447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.065719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.065986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] PATH_INFO: `/v3/projects/66bd3d3ade2a4056968f989aa79770d4/users/b9d0e99323204d13aa7b2e8387205f0b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.080940 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bd3d3ade2a4056968f989aa79770d4, user_id=b9d0e99323204d13aa7b2e8387205f0b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.084346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.097313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1150/2297] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:42 2022] PUT /identity/v3/projects/66bd3d3ade2a4056968f989aa79770d4/users/b9d0e99323204d13aa7b2e8387205f0b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:42.109225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.109225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.109508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.139113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] MFA Rules not processed for user `b9d0e99323204d13aa7b2e8387205f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:42.140310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:42.209067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1148/2298] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:42.213744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.275573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b01f79-dc12-4453-8a41-8259fedfa200 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.280157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.280806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.281064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.281331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.281964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.283821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.310495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1151/2299] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:42 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:42.314906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.325536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd08ac-0142-42e6-9ab5-5defb257cf65 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.381306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.381899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.382169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.382441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.383052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.384997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.406272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1149/2300] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:42 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:42.410806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.420990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d6d2-d04f-4bfb-8c33-c281985cfcb2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.425554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.426168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.426527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.426806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.427437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.429068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.439382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1152/2301] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:42.443498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.453348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fb5e-5476-4bc2-94c0-49f5e29d71f6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.458519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.459151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.459430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.459702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] PATH_INFO: `/v3/projects/fb157de324ca47188251223ff4a431e4/users/d5d64602d8c74d7b9b14a0370ae89581/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.474957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb157de324ca47188251223ff4a431e4, user_id=d5d64602d8c74d7b9b14a0370ae89581, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.477917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.491104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1150/2302] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:42 2022] PUT /identity/v3/projects/fb157de324ca47188251223ff4a431e4/users/d5d64602d8c74d7b9b14a0370ae89581/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:42.495116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.505187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517a4b-6dec-4819-a8e3-506fd57f634e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.561581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.562181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.562459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.562729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.563364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.565014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290caae-6f6e-4afe-a1d1-6cc0acb1592c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.574942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1153/2303] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:42.578841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.589669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efade3-8fba-4d17-b1ba-f8426eeef298 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.594260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.594897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.595190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.595483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] PATH_INFO: `/v3/projects/fb157de324ca47188251223ff4a431e4/users/d5d64602d8c74d7b9b14a0370ae89581/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.597396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb157de324ca47188251223ff4a431e4, user_id=d5d64602d8c74d7b9b14a0370ae89581, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.600959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.611292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1151/2304] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:42 2022] PUT /identity/v3/projects/fb157de324ca47188251223ff4a431e4/users/d5d64602d8c74d7b9b14a0370ae89581/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:42.621301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.621590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.621889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.641935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:42.649891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] MFA Rules not processed for user `d5d64602d8c74d7b9b14a0370ae89581`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:42.651624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:42.718516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1154/2305] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:42 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:42.729386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.731148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:42.753403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:42.789255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edb81-0891-4552-b237-1bf8bf26bbbe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.792879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.793470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.793740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.794004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.802636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.804045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.813126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1152/2306] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:42 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:42.894125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:42.904164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02d7ed-5ab1-4684-b2c5-b73cb68d77fb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:42.920763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:42.921407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:42.921682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:42.921944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:42.980191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:42.981588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b988-3d50-4165-bcb6-f1af05a62e30 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:42.990614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1155/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:47.029616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:47.094955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b3762-0b92-4afd-97ae-403cd7959ed1 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:47.099597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:47.100223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:47.100777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:47.101048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] PATH_INFO: `/v3/users/b9d0e99323204d13aa7b2e8387205f0b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:47.102163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d0e99323204d13aa7b2e8387205f0b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:47.104095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:47.151158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d0e99323204d13aa7b2e8387205f0b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.151863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d0e99323204d13aa7b2e8387205f0b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.159148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d0e99323204d13aa7b2e8387205f0b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.159770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d0e99323204d13aa7b2e8387205f0b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.167170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d0e99323204d13aa7b2e8387205f0b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.184721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1153/2308] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:47 2022] DELETE /identity/v3/users/b9d0e99323204d13aa7b2e8387205f0b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:47.189204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:47.189660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:47.189772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:47.208412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:47.209588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:47.267425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1156/2309] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:47.277136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:47.278542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:47.323599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:47.336982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e01fd-cd7f-426d-8d14-3a897c3b7e5c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:47.340663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:47.341253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:47.341526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:47.341794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:47.351498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:47.352946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:47.362310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1154/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:47.633989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:47.635426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:47.648157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9d200-f7e0-4ba3-8339-ddd21cea6891 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:47.653045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:47.653741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:47.653970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:47.654266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] PATH_INFO: `/v3/projects/66bd3d3ade2a4056968f989aa79770d4` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:47.655515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bd3d3ade2a4056968f989aa79770d4)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:47.657304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:47.696015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:47.712106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bd3d3ade2a4056968f989aa79770d4', 'request_id': 'req-961148c1-93c8-4d97-ab47-e55142f810d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.718596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bd3d3ade2a4056968f989aa79770d4', 'request_id': 'req-961148c1-93c8-4d97-ab47-e55142f810d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.728211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bd3d3ade2a4056968f989aa79770d4', 'request_id': 'req-961148c1-93c8-4d97-ab47-e55142f810d4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.740131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] The token cache is being invalidate because project 66bd3d3ade2a4056968f989aa79770d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:47.740475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.741134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.745571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1157/2311] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:47 2022] DELETE /identity/v3/projects/66bd3d3ade2a4056968f989aa79770d4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:47.749572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:47.759770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9b254-8c16-4c1c-afc6-401ce6d5d7e9 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:47.816310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:47.816941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:47.817218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:47.817507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] PATH_INFO: `/v3/users/d5d64602d8c74d7b9b14a0370ae89581` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:47.818592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d64602d8c74d7b9b14a0370ae89581)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:47.820524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:47.870173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d64602d8c74d7b9b14a0370ae89581'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.870833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d64602d8c74d7b9b14a0370ae89581'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.878009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d64602d8c74d7b9b14a0370ae89581'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.878645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d64602d8c74d7b9b14a0370ae89581'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.886081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e7b08435-c8a0-4e80-b30c-0f30be318333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d64602d8c74d7b9b14a0370ae89581'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:47.903496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1155/2312] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:47 2022] DELETE /identity/v3/users/d5d64602d8c74d7b9b14a0370ae89581 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:48.182220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.183734 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:48.199599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961148c1-93c8-4d97-ab47-e55142f810d4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.261627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.262166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.262438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.262704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] PATH_INFO: `/v3/projects/fb157de324ca47188251223ff4a431e4` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.263811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb157de324ca47188251223ff4a431e4)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.266244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.311489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb157de324ca47188251223ff4a431e4', 'request_id': 'req-242ac6bc-c162-4424-b776-d27ed1a06799'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:48.317812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb157de324ca47188251223ff4a431e4', 'request_id': 'req-242ac6bc-c162-4424-b776-d27ed1a06799'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:48.324928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb157de324ca47188251223ff4a431e4', 'request_id': 'req-242ac6bc-c162-4424-b776-d27ed1a06799'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:48.335645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] The token cache is being invalidate because project fb157de324ca47188251223ff4a431e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:48.335935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:48.336551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-242ac6bc-c162-4424-b776-d27ed1a06799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:48.338027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1158/2313] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:48 2022] DELETE /identity/v3/projects/fb157de324ca47188251223ff4a431e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:48.343043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.343504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.343600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.364405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:48.365623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:48.423404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1156/2314] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:48.431204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.431495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.431770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.451894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:48.453125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:48.461913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1159/2315] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:48.466176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.472154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:48.477211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eee925-6a62-4101-815a-148e92adda8a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.482987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.484369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.484641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.484907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.485537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:35:48.485832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.487437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.498963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1157/2316] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:35:48.503312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.512576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22585c-2908-4655-a83e-f38083e510c5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.517119 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.517699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.517973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.518260 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.518892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.521584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.547963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1160/2317] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:35:48.552103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.561829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb15f0-1e16-4a66-8283-73664f0d607d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.617055 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.617647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.617921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.618190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.618819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.620750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.641126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1158/2318] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:35:48.645785 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.655881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839bb2e-2daf-404e-ae0a-cd4354416649 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.661029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.661615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.661891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.662160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.662793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.664411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.674124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1161/2319] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:35:48.678080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.687950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13a90-f175-45ab-b9b0-2619c1e4e01b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.693299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.693886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.694167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.694441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.695074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.696710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae89c65-16d7-495d-93ab-ed86062453cd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.708837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1159/2320] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:48.712952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.724053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5025b3b-c05e-4817-aa7d-758b02692d3e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.731389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.732046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.732379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.732653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] PATH_INFO: `/v3/projects/6c4d602b4f424bdca39be44dc9c159d3/users/7347d9200d464b68a89e22e31d2e6ad4/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.735777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:48.749102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4d602b4f424bdca39be44dc9c159d3, user_id=7347d9200d464b68a89e22e31d2e6ad4, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.751741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.764888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1162/2321] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:48 2022] PUT /identity/v3/projects/6c4d602b4f424bdca39be44dc9c159d3/users/7347d9200d464b68a89e22e31d2e6ad4/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:48.777215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.777578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.777931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.806912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] MFA Rules not processed for user `7347d9200d464b68a89e22e31d2e6ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:48.808138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:48.876732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1160/2322] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:48.881665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.945426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376b8-d06f-4844-a6a4-ca428c84bc0a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:48.950298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:48.950896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.951175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.951445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.952082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:48.954023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:48.981318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1163/2323] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:48.985325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:48.995739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4a97e-4f7e-413b-9157-b26a494fcb15 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.049839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.050873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.051142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.051404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.052027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.053945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.061736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:49.075715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1161/2324] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:49.079794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.090913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e95da-7662-4b11-83fe-daa062c045bb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.095568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.096163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.096497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.096769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.098039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.099625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.110486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1164/2325] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:49.115114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.125288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eadf7-83ca-4814-87e6-49a953434922 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.130184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.130821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.131127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.131394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] PATH_INFO: `/v3/projects/ab57685e0b554493b1de355e9fb6a130/users/43022301a00542588cbf8f3cc48676bd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.146397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab57685e0b554493b1de355e9fb6a130, user_id=43022301a00542588cbf8f3cc48676bd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.149075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.162111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1162/2326] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:49 2022] PUT /identity/v3/projects/ab57685e0b554493b1de355e9fb6a130/users/43022301a00542588cbf8f3cc48676bd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:49.166065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.175829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0634e1-d650-4051-b8f4-42127181106d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.230473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.231097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.231342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.231612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.232256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.233854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae849-69d3-44e1-bf21-8b1551efdcd7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.243518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1165/2327] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:49.247642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.257909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285482-7eaa-4b14-bbcd-48771fbff222 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.262581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.263218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.263495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.263765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] PATH_INFO: `/v3/projects/ab57685e0b554493b1de355e9fb6a130/users/43022301a00542588cbf8f3cc48676bd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.265670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab57685e0b554493b1de355e9fb6a130, user_id=43022301a00542588cbf8f3cc48676bd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.268332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.279257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1163/2328] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:49 2022] PUT /identity/v3/projects/ab57685e0b554493b1de355e9fb6a130/users/43022301a00542588cbf8f3cc48676bd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:49.289991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.290355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.290627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.317559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] MFA Rules not processed for user `43022301a00542588cbf8f3cc48676bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:49.318673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:49.361477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:49.386419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1166/2329] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:49.397955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.399688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:49.459195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90de5e6-e3f1-44fe-8971-ebc294507c68 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.462857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.463457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.463730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.463999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.473561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.475040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea8c5f-1b59-4aa7-839a-3494acff3246 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.485328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1164/2330] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:49.566880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.577350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccd50b-4536-44f3-a0f6-f93f46a41220 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.580906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.581516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.582602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.582875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.638679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.640311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.650276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1167/2331] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:53.650007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:53.650574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:53.650574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:53.672402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:53.673666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:53.737593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1165/2332] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:35:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:53.791429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:53.793847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:53.873091 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f704-59bb-4735-af95-497f03262c38 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:53.878388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:53.878996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:53.879325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:53.879587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:53.889697 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:53.891275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ae7be-f9bc-4ede-af85-271ef1e13339 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:53.899929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1168/2333] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:35:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:35:54.451588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:54.526020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656b32-3452-4a57-847f-04bebeb973c5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:54.531488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:54.532221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:54.532608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:54.532909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] PATH_INFO: `/v3/users/7347d9200d464b68a89e22e31d2e6ad4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:54.534036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7347d9200d464b68a89e22e31d2e6ad4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:54.536001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:54.537687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:54.590318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7347d9200d464b68a89e22e31d2e6ad4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:54.590944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7347d9200d464b68a89e22e31d2e6ad4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:54.598650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7347d9200d464b68a89e22e31d2e6ad4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:54.599329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7347d9200d464b68a89e22e31d2e6ad4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:54.609438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7347d9200d464b68a89e22e31d2e6ad4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:54.629842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1166/2334] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:54 2022] DELETE /identity/v3/users/7347d9200d464b68a89e22e31d2e6ad4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:54.634401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:54.634688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:54.634971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:54.661346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:54.662566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:54.730787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1169/2335] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:54.741911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:54.743438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:54.810444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a79b0-45bb-4319-835d-eda452db50e7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:54.815081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:54.815721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:54.816204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:54.816700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:54.829644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:54.831124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:54.852039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1167/2336] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:55.307466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:55.313298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:55.341191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d087-c43b-487e-acb1-09a2a2252e17 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:55.349971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:55.350968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:55.351345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:55.351713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] PATH_INFO: `/v3/projects/6c4d602b4f424bdca39be44dc9c159d3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:55.353733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4d602b4f424bdca39be44dc9c159d3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:55.356037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:55.359646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:55.410902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4d602b4f424bdca39be44dc9c159d3', 'request_id': 'req-57a17d36-bf9d-4317-8b92-9af653e35eda'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.419852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4d602b4f424bdca39be44dc9c159d3', 'request_id': 'req-57a17d36-bf9d-4317-8b92-9af653e35eda'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.428644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4d602b4f424bdca39be44dc9c159d3', 'request_id': 'req-57a17d36-bf9d-4317-8b92-9af653e35eda'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.442653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] The token cache is being invalidate because project 6c4d602b4f424bdca39be44dc9c159d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:55.442981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.443696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.445270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1170/2337] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:55 2022] DELETE /identity/v3/projects/6c4d602b4f424bdca39be44dc9c159d3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:35:55.449469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:55.463617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139a5f-c07a-49a4-ab89-1c7234bb9908 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:55.535304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:55.536110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:55.536476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:55.536829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] PATH_INFO: `/v3/users/43022301a00542588cbf8f3cc48676bd` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:55.538033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43022301a00542588cbf8f3cc48676bd)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:55.540050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:55.568178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:55.598948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43022301a00542588cbf8f3cc48676bd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.601872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43022301a00542588cbf8f3cc48676bd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.611996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43022301a00542588cbf8f3cc48676bd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.617340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43022301a00542588cbf8f3cc48676bd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.629671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89e1c3a2-65a5-440b-9a8b-5aac1a15e313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43022301a00542588cbf8f3cc48676bd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:55.652026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1168/2338] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:55 2022] DELETE /identity/v3/users/43022301a00542588cbf8f3cc48676bd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:55.983055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:55.984737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:55.999984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a17d36-bf9d-4317-8b92-9af653e35eda admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.073476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.074113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.074409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.074737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] PATH_INFO: `/v3/projects/ab57685e0b554493b1de355e9fb6a130` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.075929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab57685e0b554493b1de355e9fb6a130)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.079703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.147591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab57685e0b554493b1de355e9fb6a130', 'request_id': 'req-ec7c0f22-fd01-46eb-a423-6b9dca935127'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:56.156457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab57685e0b554493b1de355e9fb6a130', 'request_id': 'req-ec7c0f22-fd01-46eb-a423-6b9dca935127'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:56.168562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab57685e0b554493b1de355e9fb6a130', 'request_id': 'req-ec7c0f22-fd01-46eb-a423-6b9dca935127'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:56.185758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] The token cache is being invalidate because project ab57685e0b554493b1de355e9fb6a130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:56.186656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:56.187162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ec7c0f22-fd01-46eb-a423-6b9dca935127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:56.193082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1171/2339] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:55 2022] DELETE /identity/v3/projects/ab57685e0b554493b1de355e9fb6a130 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:35:56.194483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.194954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.195038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.218955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:56.220194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:56.311599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1169/2340] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:56.321532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.324801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.325248 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.381503 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:56.383074 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:56.391902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1172/2341] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:56.395930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.413583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac076686-f75c-4ef9-bf4b-047ee04af335 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.426622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.427339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.427686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.427955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.428657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:35:56.428959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.430601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.443712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1170/2342] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:35:56.448424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.463912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd3195-9461-4b0e-affd-17307d26a0cd None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.469622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.470244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.470524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.470793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.471430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.474424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.500538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:56.528388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1173/2343] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:56 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:56.532044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.552683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2964e0-81ca-4e00-8d9c-1d1614f44480 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.626494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.627237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.627581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.627922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.628710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.630699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.656447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1171/2344] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:56 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:56.662066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.681049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf7f87-f027-4a04-8525-e9bb9856c832 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.710764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.711714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.712223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.712580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.713320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.715002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.737253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1174/2345] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:56.742774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.762191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910290ea-7dc4-4f8b-b0fd-eb0e79a8c592 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.767384 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.768021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.768327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.768606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.770259 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.771932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c97f9-3f31-4410-b6dd-0552de532e4e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.787900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1172/2346] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:56.789724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:56.806885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32872972-d222-496b-86e8-e25b58b9d51a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:56.811916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:56.812649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.812928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.813225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] PATH_INFO: `/v3/projects/f923901a0cb54ed8bf3d56eacfdd1b04/users/5988a1d6b83447c58864101e6c753341/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.834099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f923901a0cb54ed8bf3d56eacfdd1b04, user_id=5988a1d6b83447c58864101e6c753341, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:56.838155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:56.868301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1175/2347] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:56 2022] PUT /identity/v3/projects/f923901a0cb54ed8bf3d56eacfdd1b04/users/5988a1d6b83447c58864101e6c753341/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:56.877940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:56.879044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:56.879761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:56.904198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:56.924138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] MFA Rules not processed for user `5988a1d6b83447c58864101e6c753341`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:56.926462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:57.017904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1173/2348] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:57.022443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.108374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0c21b-14e7-4351-b223-f5927e81b89c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.114826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.115793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.116490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.116770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.117488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.119744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.149842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1176/2349] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:35:57 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:57.153546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.166473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b970932-efb4-4de3-94f7-f92c22e5d044 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.247894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.248571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.248803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.248998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.249661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.251812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.278426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1174/2350] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:35:57 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:35:57.282505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.294437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1727-bd9d-4935-9962-d2107524b9d6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.299682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.301413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.301698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.301974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.302670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.304287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.317621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1177/2351] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:35:57.321658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.334009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde899-04a5-4881-8f36-19f7a58f68ab admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.341559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.342111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.342258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.342587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] PATH_INFO: `/v3/projects/642df8949fd344dea262e7efc441d872/users/da9ece89fd0e454e9bfa9b057f628bf5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.361339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642df8949fd344dea262e7efc441d872, user_id=da9ece89fd0e454e9bfa9b057f628bf5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.364233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.382311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1175/2352] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:57 2022] PUT /identity/v3/projects/642df8949fd344dea262e7efc441d872/users/da9ece89fd0e454e9bfa9b057f628bf5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:57.386129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.399980 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1284d-ac4a-46fc-820a-b933f8ff3139 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.492692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.493325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.493609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.493883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.495508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.496841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b73e6-54b9-43a2-abeb-5e03059a7752 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.511488 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1178/2353] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:35:57.514956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.527666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb572c-1dcf-42ab-82d8-995ef6c5dae1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.535144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.535815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.536140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.536516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] PATH_INFO: `/v3/projects/642df8949fd344dea262e7efc441d872/users/da9ece89fd0e454e9bfa9b057f628bf5/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.539078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=642df8949fd344dea262e7efc441d872, user_id=da9ece89fd0e454e9bfa9b057f628bf5, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.541907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.565665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1176/2354] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:35:57 2022] PUT /identity/v3/projects/642df8949fd344dea262e7efc441d872/users/da9ece89fd0e454e9bfa9b057f628bf5/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:57.576080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.576392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.576658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.615813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] MFA Rules not processed for user `da9ece89fd0e454e9bfa9b057f628bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:57.617518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:57.641437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:57.744541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1179/2355] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:57.767271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:57.769182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:57.843294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:57.851845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62a39-1908-420d-a062-3612a7502eaa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:57.860986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:57.861718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:57.862102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:57.863422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:57.879462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:57.881897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8522119-3520-4ae5-abe2-758f1cace2ab service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:57.902351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1177/2356] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:58.424077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:58.552690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1d638-95c9-4e92-bfe3-1fd2f7500bac None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:58.558899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:58.559551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.559825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.560094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] PATH_INFO: `/v3/users/5988a1d6b83447c58864101e6c753341` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.561739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5988a1d6b83447c58864101e6c753341)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:58.563669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:58.633749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5988a1d6b83447c58864101e6c753341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:58.644168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5988a1d6b83447c58864101e6c753341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:58.646568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5988a1d6b83447c58864101e6c753341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:58.671740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5988a1d6b83447c58864101e6c753341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:58.673053 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5988a1d6b83447c58864101e6c753341'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:58.696755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1180/2357] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:58 2022] DELETE /identity/v3/users/5988a1d6b83447c58864101e6c753341 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:58.701293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.701814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.701814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.749725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:58.751769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.827750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1178/2358] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:58.851575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:58.853674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.865267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:58.867004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.945552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:59.005734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30730679-fc46-4976-ba31-3b9dffd765e7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:59.005988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc9cf-e435-451a-ba64-b40a2f882751 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:59.012497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:59.013413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:59.014403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:59.014912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:59.017319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:59.017751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:59.018012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:59.018291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:59.018626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:59.042577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:59.044003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:59.085784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1181/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:59.105367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:59.107145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:59.132032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1179/2360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:59.509370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:59.510979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:59.532798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a61894-9938-49c3-83d7-9eba51955696 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:59.551944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:59.560761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:59.561939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:59.562232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] PATH_INFO: `/v3/projects/f923901a0cb54ed8bf3d56eacfdd1b04` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:59.566801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f923901a0cb54ed8bf3d56eacfdd1b04)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:59.569640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:59.640313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f923901a0cb54ed8bf3d56eacfdd1b04', 'request_id': 'req-25b708e9-1892-41e6-a837-b26858e86b9a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.648979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f923901a0cb54ed8bf3d56eacfdd1b04', 'request_id': 'req-25b708e9-1892-41e6-a837-b26858e86b9a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.657703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f923901a0cb54ed8bf3d56eacfdd1b04', 'request_id': 'req-25b708e9-1892-41e6-a837-b26858e86b9a'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.671680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] The token cache is being invalidate because project f923901a0cb54ed8bf3d56eacfdd1b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:35:59.672102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.673922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.676527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1182/2361] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:35:59 2022] DELETE /identity/v3/projects/f923901a0cb54ed8bf3d56eacfdd1b04 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:35:59.680752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:59.705239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faa138-1999-4090-b0e5-b2a7ba1029da service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:59.740767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:59.780302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:59.781054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:59.781392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:59.781671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] PATH_INFO: `/v3/users/da9ece89fd0e454e9bfa9b057f628bf5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:59.782861 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9ece89fd0e454e9bfa9b057f628bf5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:59.785613 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:59.844946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9ece89fd0e454e9bfa9b057f628bf5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.845804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9ece89fd0e454e9bfa9b057f628bf5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.856523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9ece89fd0e454e9bfa9b057f628bf5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.857204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9ece89fd0e454e9bfa9b057f628bf5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.873478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-47beac73-77b7-4b81-b6ed-2f364a0c48f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9ece89fd0e454e9bfa9b057f628bf5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:35:59.913237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1180/2362] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:35:59 2022] DELETE /identity/v3/users/da9ece89fd0e454e9bfa9b057f628bf5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:00.290985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:00.292112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:00.325016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b708e9-1892-41e6-a837-b26858e86b9a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:00.394907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:00.395632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.395917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.396193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] PATH_INFO: `/v3/projects/642df8949fd344dea262e7efc441d872` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.400692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642df8949fd344dea262e7efc441d872)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:00.402975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:00.479373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642df8949fd344dea262e7efc441d872', 'request_id': 'req-5cf28e45-1621-4d7c-99a9-47cecc732916'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:00.489979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642df8949fd344dea262e7efc441d872', 'request_id': 'req-5cf28e45-1621-4d7c-99a9-47cecc732916'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:00.505497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642df8949fd344dea262e7efc441d872', 'request_id': 'req-5cf28e45-1621-4d7c-99a9-47cecc732916'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:00.507332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:00.523642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] The token cache is being invalidate because project 642df8949fd344dea262e7efc441d872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:00.524360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:00.524747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5cf28e45-1621-4d7c-99a9-47cecc732916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:00.526888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1183/2363] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:00 2022] DELETE /identity/v3/projects/642df8949fd344dea262e7efc441d872 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:00.531671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.531944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.532221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.558355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:00.559841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:00.651591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1181/2364] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:00.658872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.660118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.660753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.688759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:00.689978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:00.706011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1184/2365] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:00.710343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:00.721374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9e13a-0299-455e-a764-869506ab0952 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:00.727612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:00.728229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.728532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.728849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.729501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:00.729799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:00.731389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:00.746085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1182/2366] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:00.749707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:00.759916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af2175-e884-4af5-b734-6d3799da8a69 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:00.764828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:00.765430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.765704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.765976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.766613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:00.772364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:00.809078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1185/2367] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:00 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:00.810165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:00.829808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a75f-a9a8-4f6a-9cdb-e85d2e09a1f1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:00.873633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:00.912464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:00.913567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.914735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.915450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.916420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:00.918912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:00.948267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1183/2368] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:00 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:36:00.952745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:00.968818 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4763a31-cb44-4d7c-a07c-46cb2502f713 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:00.978500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:00.979365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:00.979801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:00.980237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:00.981080 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:00.982972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:00.997678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1186/2369] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:01.005471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:01.023827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88fd8c-f68b-4f14-a1bc-69454ab9fddb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:01.029227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:01.029903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.030208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.030531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.031281 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:01.032935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b6ac9-e45c-4f1a-ad3f-1ee314f0f063 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:01.048301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1184/2370] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:01.049736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:01.061325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ac7a-b767-497c-9cc1-27f66261b452 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:01.067707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:01.068403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.069021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.069021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] PATH_INFO: `/v3/projects/c16911af26a549738e0e22179b2dd3ad/users/b331fa2b141849e5b7b5dd7c7a865b24/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.123732 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16911af26a549738e0e22179b2dd3ad, user_id=b331fa2b141849e5b7b5dd7c7a865b24, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:01.126422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:01.151003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1187/2371] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:01 2022] PUT /identity/v3/projects/c16911af26a549738e0e22179b2dd3ad/users/b331fa2b141849e5b7b5dd7c7a865b24/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:36:01.156991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255941a-0c8d-4e2f-a3d9-ed60e0390bec None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.157564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255941a-0c8d-4e2f-a3d9-ed60e0390bec None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.158150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255941a-0c8d-4e2f-a3d9-ed60e0390bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.187963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-e255941a-0c8d-4e2f-a3d9-ed60e0390bec None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:01.189264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e255941a-0c8d-4e2f-a3d9-ed60e0390bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:01.276282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1185/2372] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:01.287255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:01.289045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:01.368603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b8386-ba33-4ad6-b2de-b9c5dc106590 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:01.373673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:01.374392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.374962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.375324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.388676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:01.390414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:01.405531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1188/2373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:02.401506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.402089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.402292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.460669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] MFA Rules not processed for user `b331fa2b141849e5b7b5dd7c7a865b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:02.462971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:02.600191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1186/2374] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:02.600982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.627165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb0230-44e7-454a-a992-4fd42ec4f007 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:02.633224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:02.633964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.634356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.634900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.635808 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:02.637822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:02.687718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1189/2375] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:02 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:02.692383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.704034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eba9f2-07ed-4651-8831-5383b2d0c7be None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:02.755852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:02.778879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:02.779496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.779846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.780228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.781013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:02.783237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:02.809911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1187/2376] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:02 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:36:02.813831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.828662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921360c-b1e1-4e64-b2c6-727d696aacb4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:02.835555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:02.836422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.837419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.837828 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.839087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:02.841421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:02.857632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1190/2377] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:36:02.861573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.872769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951e95-1bb3-4e33-b788-9ab8d92e8d8b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:02.877666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:02.878576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.879017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.879502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] PATH_INFO: `/v3/projects/f9528fdb628249778376b8dc8a96e7b7/users/10cd498df05e4fe2b318ce355c77b60a/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.901129 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9528fdb628249778376b8dc8a96e7b7, user_id=10cd498df05e4fe2b318ce355c77b60a, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:02.903712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f9da-4981-4022-a51a-53d6735247c7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:02.938849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1188/2378] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:02 2022] PUT /identity/v3/projects/f9528fdb628249778376b8dc8a96e7b7/users/10cd498df05e4fe2b318ce355c77b60a/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:02.943826 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.960356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478cd95-7c24-4a9b-bce0-cecef563c0c5 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:03.067222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f2d97-b330-4c03-8de6-ee38303926a5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:03.067457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f2d97-b330-4c03-8de6-ee38303926a5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:03.067582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f2d97-b330-4c03-8de6-ee38303926a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:03.078399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:03.100664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:03.103217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:03.103217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:03.103217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:03.103217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:03.104325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2d1f2d97-b330-4c03-8de6-ee38303926a5 None None] MFA Rules not processed for user `30ccb163613a40ac8042f229199717f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:03.105044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:03.113502 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2d1f2d97-b330-4c03-8de6-ee38303926a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:03.123400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1191/2379] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:36:03.129001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:03.143207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58e8fc-a447-4d9d-8fd6-7cb6eec669c4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:03.156619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:03.157452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:03.157789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:03.158120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] PATH_INFO: `/v3/projects/f9528fdb628249778376b8dc8a96e7b7/users/10cd498df05e4fe2b318ce355c77b60a/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:03.161521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9528fdb628249778376b8dc8a96e7b7, user_id=10cd498df05e4fe2b318ce355c77b60a, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:03.164274 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:03.184339 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1192/2380] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:03 2022] PUT /identity/v3/projects/f9528fdb628249778376b8dc8a96e7b7/users/10cd498df05e4fe2b318ce355c77b60a/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:03.212643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1189/2381] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:03.737325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:03.835041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b34d9-b70c-441e-843c-41dac22c0188 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:03.839601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:03.840350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:03.840898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:03.841231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:03.843457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:03.857003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:03.859095 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:03.869117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1193/2382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:04.166184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:04.166869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:04.166869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:04.239343 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] MFA Rules not processed for user `10cd498df05e4fe2b318ce355c77b60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:04.243141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:04.388832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1190/2383] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:04.402325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:04.405106 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:04.547429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd281c-733c-42cc-88c0-a0fb2dc07480 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:04.556002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:04.557017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:04.557474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:04.558117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:04.634419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:04.635953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9087d5-04b4-4aae-ac00-ba568ad4cfe6 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:04.649128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1194/2384] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:06.149422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:06.150823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:06.162794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf197f-d686-40f6-b094-30e786e9b692 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:06.166389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:06.166986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:06.167265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:06.167536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:06.176376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:06.177799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:06.187152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1191/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:36:12.666203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:12.666683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:12.666801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:12.692276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:12.693592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:12.727618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:12.762460 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1195/2386] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:12.777784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:12.779705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:12.820050 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:12.871136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a396ec-878f-4f63-b2d6-0a4cd2b8d1b3 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:12.876614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:12.877991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:12.878419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:12.878698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:12.889020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:12.891334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:12.900665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1192/2387] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:36:20.303838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:20.305644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:20.378845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:20.409579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015fa76-d6de-44f0-a085-8cc0d82c6e03 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:20.416747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:20.417415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:20.417693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:20.417963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:20.420854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:20.437672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:20.439572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:20.452407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1196/2388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:20.474596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0101a-002a-4db9-98e1-f071dde9f7db service swift] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:20.479553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:20.480378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:20.480657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:20.480923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] PATH_INFO: `/v3/users/b331fa2b141849e5b7b5dd7c7a865b24` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:20.482031 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b331fa2b141849e5b7b5dd7c7a865b24)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:20.483927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:20.545900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b331fa2b141849e5b7b5dd7c7a865b24'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:20.546647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b331fa2b141849e5b7b5dd7c7a865b24'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:20.557425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b331fa2b141849e5b7b5dd7c7a865b24'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:20.558068 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b331fa2b141849e5b7b5dd7c7a865b24'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:20.566176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b331fa2b141849e5b7b5dd7c7a865b24'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:20.614595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1193/2389] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:20 2022] DELETE /identity/v3/users/b331fa2b141849e5b7b5dd7c7a865b24 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:20.825942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:20.844549 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd1b7-c97d-49f6-87dd-890b541c1534 service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:20.932065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:20.932876 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:20.933232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:20.933650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] PATH_INFO: `/v3/projects/c16911af26a549738e0e22179b2dd3ad` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:20.935140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16911af26a549738e0e22179b2dd3ad)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:20.937514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:20.999654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16911af26a549738e0e22179b2dd3ad', 'request_id': 'req-7ce6716c-ef60-464c-b88a-60c8195156a4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.007153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16911af26a549738e0e22179b2dd3ad', 'request_id': 'req-7ce6716c-ef60-464c-b88a-60c8195156a4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.015862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16911af26a549738e0e22179b2dd3ad', 'request_id': 'req-7ce6716c-ef60-464c-b88a-60c8195156a4'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.024371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:21.030071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] The token cache is being invalidate because project c16911af26a549738e0e22179b2dd3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:21.030601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.031537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.034457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1197/2390] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:20 2022] DELETE /identity/v3/projects/c16911af26a549738e0e22179b2dd3ad => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:21.038297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.039785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:21.053968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baabb2-753e-4a63-8e85-fa969cbbb451 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.093097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:21.122634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.123271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.123552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.123829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] PATH_INFO: `/v3/users/10cd498df05e4fe2b318ce355c77b60a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.124995 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10cd498df05e4fe2b318ce355c77b60a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.127132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.182783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10cd498df05e4fe2b318ce355c77b60a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.183479 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10cd498df05e4fe2b318ce355c77b60a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.191328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10cd498df05e4fe2b318ce355c77b60a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.191997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10cd498df05e4fe2b318ce355c77b60a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.199418 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3f77bde3-fb7d-44b1-bec4-9599b5e51e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10cd498df05e4fe2b318ce355c77b60a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.218748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1194/2391] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:21 2022] DELETE /identity/v3/users/10cd498df05e4fe2b318ce355c77b60a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:21.391946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.394228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:21.409473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6716c-ef60-464c-b88a-60c8195156a4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.472172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.472831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.473104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.473370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] PATH_INFO: `/v3/projects/f9528fdb628249778376b8dc8a96e7b7` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.474731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9528fdb628249778376b8dc8a96e7b7)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.476689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.521294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9528fdb628249778376b8dc8a96e7b7', 'request_id': 'req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.527334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9528fdb628249778376b8dc8a96e7b7', 'request_id': 'req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.535187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9528fdb628249778376b8dc8a96e7b7', 'request_id': 'req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.545283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] The token cache is being invalidate because project f9528fdb628249778376b8dc8a96e7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:21.545566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.546179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-77de7e76-c5ab-4fb2-a34b-ec4d29027cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:21.547636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1198/2392] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:21 2022] DELETE /identity/v3/projects/f9528fdb628249778376b8dc8a96e7b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:21.553632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.554156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.554262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.573744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:21.574929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:21.633282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1195/2393] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:21.641125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.641408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.641707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.663304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:21.664543 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:21.672523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1199/2394] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:21.676810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.686506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36611ab1-b3d5-4981-af22-9987eb6a79e1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.691747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.692377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.692655 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.692926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.693589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:21.693888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.695470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.707566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1196/2395] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:21.711653 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.720786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5840113-687a-438c-ac0b-52b514e3cbf2 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.725599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.726137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.726396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.726947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.727606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.729673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.757741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1200/2396] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:21 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:21.762501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.779237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfa14a-3294-4c4d-b277-85ff73ec5049 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.821053 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:21.838815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.839414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.839686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.839952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.840623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.842507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.864737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1197/2397] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:21 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:21.869008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.879665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562c8d-515e-4ba6-b401-668c2d4d7a3d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.884193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.884829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.885105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.885378 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.886002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.887573 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.898425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1201/2398] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:21.902294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.912447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617830e6-df79-43c4-a19c-d837aa560863 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.917047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.917635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.917913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.918183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.918815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.920440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15268d98-efe7-4864-9ba7-d96286388b9b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.931052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1198/2399] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:21.935086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:21.945954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570c32c-2c22-4238-a1a8-dac962fbdc23 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:21.956457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:21.957112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:21.957385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:21.957652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] PATH_INFO: `/v3/projects/68d09451a7774b63bc312bf0de415e01/users/f2f12856ec9045ef804d6a54134e7f22/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:21.972971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d09451a7774b63bc312bf0de415e01, user_id=f2f12856ec9045ef804d6a54134e7f22, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:21.975983 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:21.989942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1202/2400] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:21 2022] PUT /identity/v3/projects/68d09451a7774b63bc312bf0de415e01/users/f2f12856ec9045ef804d6a54134e7f22/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:22.000124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6261ce-013d-4232-b7ee-2cada5e1bf61 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:22.000443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6261ce-013d-4232-b7ee-2cada5e1bf61 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:22.001190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6261ce-013d-4232-b7ee-2cada5e1bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:22.032229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9a6261ce-013d-4232-b7ee-2cada5e1bf61 None None] MFA Rules not processed for user `f2f12856ec9045ef804d6a54134e7f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:22.033779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9a6261ce-013d-4232-b7ee-2cada5e1bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:22.102957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1199/2401] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:22.116059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:22.117861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:22.150536 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:22.178819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4840-8407-4598-9f1f-c60672bc6fa8 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:22.182687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:22.183288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:22.183564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:22.183859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:22.192707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:22.194111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:22.202626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1203/2402] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:28.901325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:28.901835 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:28.901934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:28.921550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:28.922796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:28.979952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1200/2403] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:28.994187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:28.995606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:29.068497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851819-ae6c-4111-8b7f-8c254c664e08 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:29.074003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:29.074635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:29.074905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:29.075169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:29.084950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:29.086603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:29.097504 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1204/2404] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:36:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:36:38.973139 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:39.093003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=46 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:41.863941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:41.865424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:41.921893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751eef1-7623-4b1b-a2b2-a17567d562b9 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:41.926667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:41.927276 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:41.927556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:41.927827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:41.929768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:41.995056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:41.996532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:42.006501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1201/2405] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:45.255971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.257584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.274428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9ea3-044a-4229-97c6-1232ab8135c0 service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.279017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.279642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.279951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.280220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] PATH_INFO: `/v3/users/48e588ff6d5a4a37b902abadfb6d04fc` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.281400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e588ff6d5a4a37b902abadfb6d04fc)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:45.283323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:45.330044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e588ff6d5a4a37b902abadfb6d04fc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.337686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e588ff6d5a4a37b902abadfb6d04fc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.338446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e588ff6d5a4a37b902abadfb6d04fc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.346195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e588ff6d5a4a37b902abadfb6d04fc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.346829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-1d7c6fe9-6c8b-4a93-9a88-e94e1bfbd609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e588ff6d5a4a37b902abadfb6d04fc'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.365441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1205/2406] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:45 2022] DELETE /identity/v3/users/48e588ff6d5a4a37b902abadfb6d04fc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:45.543749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.555603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2118b-15aa-4f12-823f-7c6d35306e57 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.614174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.614803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.615078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.615351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] PATH_INFO: `/v3/projects/fd13dd8ec810435491e3f3cba7cca4d1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.616496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd13dd8ec810435491e3f3cba7cca4d1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:45.618647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:45.667920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd13dd8ec810435491e3f3cba7cca4d1', 'request_id': 'req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.670282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:45.675810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd13dd8ec810435491e3f3cba7cca4d1', 'request_id': 'req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.682598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd13dd8ec810435491e3f3cba7cca4d1', 'request_id': 'req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.694326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] The token cache is being invalidate because project fd13dd8ec810435491e3f3cba7cca4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:45.694620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.695280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4a081937-89cc-4ed3-ab99-5f9f5d1fc6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:45.696979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1202/2407] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:45 2022] DELETE /identity/v3/projects/fd13dd8ec810435491e3f3cba7cca4d1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:45.702161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.702450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.702726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.722862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:45.724609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.783987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1206/2408] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:45.791918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.792200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.792506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.825169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:45.826374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.835007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1203/2409] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:45.842267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.852801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b1875-b415-4739-8c3c-f61e3e73a07a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.857576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.857977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.858256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.858529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.859160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:45.859577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:45.861723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:45.873736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1207/2410] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:45.878483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.887847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e1062-659f-4339-aa64-c5849d41f655 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.892918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.893529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.893785 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.894051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.894671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:45.896577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:45.927729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1204/2411] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:45 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:45.931698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.941751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17cf5-4b66-4928-aff2-a1f8f8e90e57 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.997722 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.998331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.998605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.998872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.999515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.001492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.009534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:46.024132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1208/2412] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:45 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:46.029026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.040396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68312e1-fefe-4da7-b842-f0472027d1d5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.045130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.045733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.046012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.046283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.047534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.049248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.059085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1205/2413] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:46.063295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.073181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef94cc-4bbb-4943-b6c7-a4ebbd92d7b1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.077812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.078891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.079169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.079548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.080368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.081950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea36317-9122-486d-b7b9-81a80a790d4f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.097929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1209/2414] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:46.100008 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.110457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51dbf8-31da-4e51-98b3-b24f122ef3de admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.115041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.115679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.115951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.116217 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] PATH_INFO: `/v3/projects/16145413c8814759ae5d24a90c4179ae/users/5467a4844f3f49a18f0a52f76146072b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.132620 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16145413c8814759ae5d24a90c4179ae, user_id=5467a4844f3f49a18f0a52f76146072b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.137140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.154110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1206/2415] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:46 2022] PUT /identity/v3/projects/16145413c8814759ae5d24a90c4179ae/users/5467a4844f3f49a18f0a52f76146072b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:46.165030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.165613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.166402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.194368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] MFA Rules not processed for user `5467a4844f3f49a18f0a52f76146072b`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:46.195571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:46.268426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1210/2416] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:46.282639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.284533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:46.360295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e7c27-94ac-4a0e-aae1-a8be6d0b9a49 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.365908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.366507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.366780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.367048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.376388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.377987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134faaf-7424-4aa0-b953-e463c5161f6b service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.387553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1207/2417] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:46.597916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.662893 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39fa7-a04e-4233-a8c3-2d167ca76b61 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.667518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.668158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.668467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.668744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] PATH_INFO: `/v3/users/5467a4844f3f49a18f0a52f76146072b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.669875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5467a4844f3f49a18f0a52f76146072b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.671798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.719683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5467a4844f3f49a18f0a52f76146072b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:46.726988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5467a4844f3f49a18f0a52f76146072b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:46.727608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5467a4844f3f49a18f0a52f76146072b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:46.735583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5467a4844f3f49a18f0a52f76146072b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:46.736315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5467a4844f3f49a18f0a52f76146072b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:46.742281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:46.757019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1211/2418] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:46 2022] DELETE /identity/v3/users/5467a4844f3f49a18f0a52f76146072b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:46.761600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.762094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.762226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.782440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:46.783646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:46.844135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1208/2419] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:46.854230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:46.855620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:46.916209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d36e3-3c54-4a22-8cee-dbc1b10d7a83 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:46.919922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:46.920562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:46.920844 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:46.921118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:46.930918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.932368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef39a-6f6f-48ef-ae9b-2041576cc32c service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.942593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1212/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:47.225619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.227219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:47.238003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:47.245604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591bae3-4a24-4c1c-842e-cc2a075d45f7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.252442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.253237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.253478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.253813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] PATH_INFO: `/v3/projects/16145413c8814759ae5d24a90c4179ae` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.255036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16145413c8814759ae5d24a90c4179ae)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.257179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.306983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16145413c8814759ae5d24a90c4179ae', 'request_id': 'req-cf859801-ba47-4a3d-b050-220a22285562'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:47.314667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16145413c8814759ae5d24a90c4179ae', 'request_id': 'req-cf859801-ba47-4a3d-b050-220a22285562'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:47.320816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16145413c8814759ae5d24a90c4179ae', 'request_id': 'req-cf859801-ba47-4a3d-b050-220a22285562'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:47.338344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] The token cache is being invalidate because project 16145413c8814759ae5d24a90c4179ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:47.338627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:47.339283 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-cf859801-ba47-4a3d-b050-220a22285562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:47.340869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1209/2421] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:47 2022] DELETE /identity/v3/projects/16145413c8814759ae5d24a90c4179ae => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:47.346444 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.346930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.347056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.367486 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:47.368723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:47.428334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1213/2422] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:47.437551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.437950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.438263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.458726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:47.459953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:47.467656 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1210/2423] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:47.472784 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.482924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2889f68-efa9-4851-b6c0-a40971a59882 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.489781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.490389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.490663 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.490950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.491598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:47.491897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.493548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.506964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1214/2424] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:47.511341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.520779 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a07d51-ec2d-4c8d-b198-9156845487dc None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.525435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.526026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.526304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.526573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.527204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.529136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.557503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1211/2425] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:47.562079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.572723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71083ef-b688-4073-8126-c2e825b0ef58 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.629815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.630429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.630710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.630982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.631631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.633618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.654778 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1215/2426] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:36:47.659477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.669915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4d3aa-a31c-4b30-adc4-491801e357f7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.674521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.675108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.675383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.675654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.676299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.677885 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.689677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1212/2427] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:47.692636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.702591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e97fe-581b-41f7-8d58-a804f76cbcae admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.707636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.708331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.708651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.708994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.709806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.711548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541414f-75c9-48cd-a011-803b45ea9f79 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.712829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:47.724571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1216/2428] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:47.729080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.739232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe17aa-c2ed-4f84-83f1-9ea7dd6d028d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.743869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.744548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.744833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.745115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] PATH_INFO: `/v3/projects/d3acca56936e4450bf490548c8016192/users/07f05ceac94f4cc5bc79878691f1f0fa/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.759457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3acca56936e4450bf490548c8016192, user_id=07f05ceac94f4cc5bc79878691f1f0fa, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.762200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:47.777515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1213/2429] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:47 2022] PUT /identity/v3/projects/d3acca56936e4450bf490548c8016192/users/07f05ceac94f4cc5bc79878691f1f0fa/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:47.788560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.788846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.789144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.825160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] MFA Rules not processed for user `07f05ceac94f4cc5bc79878691f1f0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:47.826435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:47.898897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1217/2430] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:47.903648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:47.966393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf780f-8c43-4905-8f59-245670e7a47c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:47.971543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:47.972154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.972588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.972932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.973841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:47.975756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.002850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1214/2431] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:48.007018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.018733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc2d09-2793-44af-a79b-7b4e2e22133e None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.076219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.076757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.077033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.077299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.077926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.081229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.109087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1218/2432] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:48 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:36:48.111540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.123027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff0d-bda5-45ac-b9e9-470497eba338 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.128684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.129288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.129561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.129829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.130459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.132032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.133448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:48.147152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1215/2433] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:48.150640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.160834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762b6d0-91da-431f-bb54-9b5f2b31699d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.165911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.166578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.166854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.167125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] PATH_INFO: `/v3/projects/ccd328929aa24d1c855039b1b4decbc3/users/790b9c09ff634fcf89417850eae7a244/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.186911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd328929aa24d1c855039b1b4decbc3, user_id=790b9c09ff634fcf89417850eae7a244, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.190345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.203070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1219/2434] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:48 2022] PUT /identity/v3/projects/ccd328929aa24d1c855039b1b4decbc3/users/790b9c09ff634fcf89417850eae7a244/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:48.207678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.217642 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacad133-8e10-4ff2-bdbb-5930b80254fd admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.275749 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.276371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.276645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.276912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.277569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.279142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00a927-3cf4-45bd-a32a-c5524e3ee06d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.288688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1216/2435] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:48.292945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.304453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49729f-de85-4a1c-a235-aad03a1c6e9c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.309059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.309714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.309985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.310251 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] PATH_INFO: `/v3/projects/ccd328929aa24d1c855039b1b4decbc3/users/790b9c09ff634fcf89417850eae7a244/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.312146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd328929aa24d1c855039b1b4decbc3, user_id=790b9c09ff634fcf89417850eae7a244, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.315532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.326459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1220/2436] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:48 2022] PUT /identity/v3/projects/ccd328929aa24d1c855039b1b4decbc3/users/790b9c09ff634fcf89417850eae7a244/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:36:48.336568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.336860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.337143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.367332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] MFA Rules not processed for user `790b9c09ff634fcf89417850eae7a244`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:48.373809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:48.443038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1217/2437] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:48.454899 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.456693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:48.500797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:48.518807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8c0-530c-4eca-a089-7047d7407fa1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.522531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.523133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.523412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.523683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.533498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.535009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17eca4-b792-49b3-9400-eabc113bfc6b service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.544329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1221/2438] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:48.978295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:49.044873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a2e34-d002-4753-a37b-8e03976f4f56 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:49.050210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:49.050849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:49.051126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:49.051392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] PATH_INFO: `/v3/users/07f05ceac94f4cc5bc79878691f1f0fa` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:49.052550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f05ceac94f4cc5bc79878691f1f0fa)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:49.054463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:49.075831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:49.101293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f05ceac94f4cc5bc79878691f1f0fa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.101866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f05ceac94f4cc5bc79878691f1f0fa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.115567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f05ceac94f4cc5bc79878691f1f0fa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.116190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f05ceac94f4cc5bc79878691f1f0fa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.123619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f05ceac94f4cc5bc79878691f1f0fa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.141157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1218/2439] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:48 2022] DELETE /identity/v3/users/07f05ceac94f4cc5bc79878691f1f0fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:49.145633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:49.145920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:49.146196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:49.165502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:49.167033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:49.227375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1222/2440] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:49.238410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:49.240007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:49.296574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac63dad-ab9b-4ea2-a13e-49c075180ae4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:49.300745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:49.301341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:49.301618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:49.301891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:49.310910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:49.312933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:49.323305 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1219/2441] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:36:49.590037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:49.591499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:49.604506 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464718-c755-4e50-8f90-3754a4d5c2df None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:49.609993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:49.610857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:49.611133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:49.611394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] PATH_INFO: `/v3/projects/d3acca56936e4450bf490548c8016192` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:49.612659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3acca56936e4450bf490548c8016192)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:49.615125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:49.660976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3acca56936e4450bf490548c8016192', 'request_id': 'req-ef945330-d0ba-4050-9473-7f7f2fc553a6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.667047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3acca56936e4450bf490548c8016192', 'request_id': 'req-ef945330-d0ba-4050-9473-7f7f2fc553a6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.674748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3acca56936e4450bf490548c8016192', 'request_id': 'req-ef945330-d0ba-4050-9473-7f7f2fc553a6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.686463 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] The token cache is being invalidate because project d3acca56936e4450bf490548c8016192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:49.686752 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.687570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.689377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1223/2442] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:49 2022] DELETE /identity/v3/projects/d3acca56936e4450bf490548c8016192 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:49.693336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:49.702927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154cecd-46cd-4ece-ae88-445c0b7ee5d7 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:49.761331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:49.761967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:49.762242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:49.762512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] PATH_INFO: `/v3/users/790b9c09ff634fcf89417850eae7a244` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:49.763618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] RBAC: Authorizing `identity:delete_user(user_id=790b9c09ff634fcf89417850eae7a244)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:49.765617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:49.806892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:49.814019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790b9c09ff634fcf89417850eae7a244'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.814958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790b9c09ff634fcf89417850eae7a244'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.823005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790b9c09ff634fcf89417850eae7a244'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.823645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790b9c09ff634fcf89417850eae7a244'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.831636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ac835681-30af-4810-9803-b97cfb1f229f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790b9c09ff634fcf89417850eae7a244'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:49.849582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1220/2443] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:49 2022] DELETE /identity/v3/users/790b9c09ff634fcf89417850eae7a244 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:50.135166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.136299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:50.151351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945330-d0ba-4050-9473-7f7f2fc553a6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.209853 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.210403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.210678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.210943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] PATH_INFO: `/v3/projects/ccd328929aa24d1c855039b1b4decbc3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.212037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd328929aa24d1c855039b1b4decbc3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.214199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.220332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:50.260895 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd328929aa24d1c855039b1b4decbc3', 'request_id': 'req-e2737439-1f97-4000-81ae-d190008155cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:50.267707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd328929aa24d1c855039b1b4decbc3', 'request_id': 'req-e2737439-1f97-4000-81ae-d190008155cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:50.275222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd328929aa24d1c855039b1b4decbc3', 'request_id': 'req-e2737439-1f97-4000-81ae-d190008155cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:50.285525 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] The token cache is being invalidate because project ccd328929aa24d1c855039b1b4decbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:50.285809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:50.286452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e2737439-1f97-4000-81ae-d190008155cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:50.287929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1224/2444] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:50 2022] DELETE /identity/v3/projects/ccd328929aa24d1c855039b1b4decbc3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:50.292849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.293307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.293404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.313574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:50.314778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:50.374365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1221/2445] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:50.384162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.384480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.384760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.405457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:50.407009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:50.414878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1225/2446] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:50.419112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.428817 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e1d17-daee-442f-b859-9761aaf9c5db None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.433389 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.433988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.434267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.434547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.435192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:50.435494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.437157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.449285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1222/2447] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:50.454421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.463947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384170f1-7b28-4cc2-9718-ea17d83f975a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.468912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.469604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.469885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.470159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.470798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.472759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.511343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1226/2448] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:50.511622 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.522837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a887a4-770f-4601-b556-5638c51dd17a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.580722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.581334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.581617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.581890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.582517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.584469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.610027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1223/2449] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:50.614059 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.624394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4ddae-03a9-4208-81f1-4463fbc7f47d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.629726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.630320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.630596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.630866 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.631502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.633103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.643542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1227/2450] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:50.647460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.657266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672708b-ec91-45cd-bc13-f8bb1c93a84e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.661857 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.662437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.662713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.662982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.663605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.665227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30edc7-08de-427a-a291-3c74f4d23e33 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.677164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1224/2451] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:50.681185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.691779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd59ffd-7aed-4760-8f27-5f79a63f5770 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.696861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.697505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.697787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.698065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] PATH_INFO: `/v3/projects/0c423eafcb414dc19fa8ca71fa523193/users/643c61fb9258490f888fba56a9415405/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.712750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c423eafcb414dc19fa8ca71fa523193, user_id=643c61fb9258490f888fba56a9415405, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.715322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.728326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1228/2452] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:50 2022] PUT /identity/v3/projects/0c423eafcb414dc19fa8ca71fa523193/users/643c61fb9258490f888fba56a9415405/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:50.741774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26db5-703e-4407-8b50-a346436764da None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.742060 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26db5-703e-4407-8b50-a346436764da None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.742333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26db5-703e-4407-8b50-a346436764da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.770893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-8ec26db5-703e-4407-8b50-a346436764da None None] MFA Rules not processed for user `643c61fb9258490f888fba56a9415405`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:50.772087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-8ec26db5-703e-4407-8b50-a346436764da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:50.786002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:50.841504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1225/2453] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:50.846441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.915445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c41449-f7c9-4144-aebb-b9b4e07c1ca4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:50.920131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:50.920758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.921036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.921308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.922716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:50.924642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:50.950574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1229/2454] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:50.955036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec26db5-703e-4407-8b50-a346436764da None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:50.965677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec26db5-703e-4407-8b50-a346436764da None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.020714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.021309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.021583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.021850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.022462 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.024372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.045362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1226/2455] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:36:51.050341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.060419 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90bd78-fa15-4ba9-8a35-87d29f23439c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.065186 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.065776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.066047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.066320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.066949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.068634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.079154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1230/2456] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:51.083160 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.093074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8700e62-cd90-44c4-ac09-358a5664e4b7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.098915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.099548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.099822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.100092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] PATH_INFO: `/v3/projects/7cdad627b89448f28b2e7756c7d5bb32/users/e52cd757f66747f499c099149f695c01/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.115524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdad627b89448f28b2e7756c7d5bb32, user_id=e52cd757f66747f499c099149f695c01, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.118209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.130560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1227/2457] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:51 2022] PUT /identity/v3/projects/7cdad627b89448f28b2e7756c7d5bb32/users/e52cd757f66747f499c099149f695c01/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:51.134667 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.141010 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:51.145428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13534dd-1f66-41ea-9a41-c98a265d2332 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.206586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.207197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.207472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.207740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.208407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.210017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945eb541-6135-4049-94e9-71c85d017951 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.220235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1231/2458] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:51.224734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.235429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7291-083b-43f9-b1ce-2b2fab5dc6e5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.240040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.240737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.241016 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.241290 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] PATH_INFO: `/v3/projects/7cdad627b89448f28b2e7756c7d5bb32/users/e52cd757f66747f499c099149f695c01/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.243183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdad627b89448f28b2e7756c7d5bb32, user_id=e52cd757f66747f499c099149f695c01, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.245855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.255858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1228/2459] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:51 2022] PUT /identity/v3/projects/7cdad627b89448f28b2e7756c7d5bb32/users/e52cd757f66747f499c099149f695c01/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:51.266763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.267045 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.267319 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.295900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] MFA Rules not processed for user `e52cd757f66747f499c099149f695c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:51.297737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.364987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1232/2460] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:51.376270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.378024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.434189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43537b-8bf5-4a10-bdfe-71c9f8bd9007 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.437877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.438464 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.438736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.439003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.447973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.449424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.459223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1229/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:51.534799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.536165 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.547985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3522d7-748d-4a6c-835d-06998d09baf0 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.551583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.552177 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.552477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.552743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.608223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.609795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b109f1-2bc1-4c33-b9ed-7dbaece75dfe service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.618465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1233/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:54.366550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:54.376216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:54.428636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b507c-9117-47ee-b53b-9e1c05a851dd service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:54.433146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:54.433781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.434057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.434325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] PATH_INFO: `/v3/users/643c61fb9258490f888fba56a9415405` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.435436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=643c61fb9258490f888fba56a9415405)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:54.438101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:54.482319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643c61fb9258490f888fba56a9415405'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:54.482960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643c61fb9258490f888fba56a9415405'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:54.490328 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643c61fb9258490f888fba56a9415405'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:54.491067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643c61fb9258490f888fba56a9415405'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:54.497796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643c61fb9258490f888fba56a9415405'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:54.514741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1230/2463] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:54 2022] DELETE /identity/v3/users/643c61fb9258490f888fba56a9415405 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:54.519214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.519718 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.519831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.538649 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:54.539823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.558526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:54.598771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1234/2464] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:54.608350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:54.610245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.673812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5904bd9-46fe-4758-b489-a81663ab54ec admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:54.677371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:54.677971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.678249 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.678523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.687614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:54.689183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:54.700102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1231/2465] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:54.841531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:54.845645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.858727 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b25c-8028-4002-b3ab-072b7117186d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:54.864104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:54.864755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.865155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.865426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.867152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.927541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:54.928968 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:54.942911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1235/2466] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:55.025282 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:55.026850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:55.040107 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aaf44-054e-4b8d-8910-01f006cb6c30 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:55.045705 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:55.046463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:55.047927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:55.047927 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] PATH_INFO: `/v3/projects/0c423eafcb414dc19fa8ca71fa523193` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:55.049233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c423eafcb414dc19fa8ca71fa523193)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:55.051152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:55.092244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:55.113769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c423eafcb414dc19fa8ca71fa523193', 'request_id': 'req-d0138d8f-d020-49be-b28c-c419838c944d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.121218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c423eafcb414dc19fa8ca71fa523193', 'request_id': 'req-d0138d8f-d020-49be-b28c-c419838c944d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.127145 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c423eafcb414dc19fa8ca71fa523193', 'request_id': 'req-d0138d8f-d020-49be-b28c-c419838c944d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.138568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] The token cache is being invalidate because project 0c423eafcb414dc19fa8ca71fa523193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:55.138934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.139632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.141973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1232/2467] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:55 2022] DELETE /identity/v3/projects/0c423eafcb414dc19fa8ca71fa523193 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:55.145946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:55.156666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2413a-5d39-43ef-9459-e54389fc321d service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:55.216458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:55.217126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:55.217430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:55.217729 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] PATH_INFO: `/v3/users/e52cd757f66747f499c099149f695c01` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:55.219351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52cd757f66747f499c099149f695c01)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:55.221365 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:55.273870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52cd757f66747f499c099149f695c01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.281780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52cd757f66747f499c099149f695c01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.282424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52cd757f66747f499c099149f695c01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.289731 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52cd757f66747f499c099149f695c01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.290390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3c600c0d-f4b4-4e02-a662-18e36f980230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52cd757f66747f499c099149f695c01'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.292299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:55.310851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1236/2468] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:36:55 2022] DELETE /identity/v3/users/e52cd757f66747f499c099149f695c01 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:55.742542 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:55.744307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:55.758140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8f-d020-49be-b28c-c419838c944d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:55.841596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:55.842335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:55.842712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:55.843082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] PATH_INFO: `/v3/projects/7cdad627b89448f28b2e7756c7d5bb32` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:55.844649 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cdad627b89448f28b2e7756c7d5bb32)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:55.847230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:55.911791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cdad627b89448f28b2e7756c7d5bb32', 'request_id': 'req-07f49783-e5ce-4ec0-945f-75973aed1234'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.935833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cdad627b89448f28b2e7756c7d5bb32', 'request_id': 'req-07f49783-e5ce-4ec0-945f-75973aed1234'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.954003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cdad627b89448f28b2e7756c7d5bb32', 'request_id': 'req-07f49783-e5ce-4ec0-945f-75973aed1234'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.971159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] The token cache is being invalidate because project 7cdad627b89448f28b2e7756c7d5bb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:36:55.972152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.973888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-07f49783-e5ce-4ec0-945f-75973aed1234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:36:55.976163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1233/2469] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:36:55 2022] DELETE /identity/v3/projects/7cdad627b89448f28b2e7756c7d5bb32 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:55.981242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:55.981706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:55.982054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.063409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:56.076664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:56.184410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1237/2470] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:56.192983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.193254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.193531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.221039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:56.222496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:56.232521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1234/2471] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:56.236830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.248317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba56d1-bdfa-44bc-8906-19123e744fd1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.259848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.260521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.260804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.261085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.263046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:36:56.263362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.265077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.282109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1238/2472] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:36:56.283811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.298675 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61d29-df90-4dfe-800c-1e9928906615 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.305391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.306097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.306437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.306962 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.307719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.310799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.360296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1235/2473] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:56.364181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.377008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca1fe6-f2f6-4d90-a2a5-95a996e320ca admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.470399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.471144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.471505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.471848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.472607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.475225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.507602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1239/2474] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:56.508780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.520812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133db12-b315-4995-ac66-0e31256bebc7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.525969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.526821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.527169 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.527463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.528144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.529780 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.541377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1236/2475] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:36:56.545652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.557196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69fd41-2d22-4b67-ae96-2a43fc0800d6 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.563111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.563832 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.564116 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.564434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.565076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.566672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54fa1d-e808-4e25-9176-e0b036f1a4ec admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.577678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1240/2476] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:56.582113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.588916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:56.592800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b4d3d-0394-4e94-a61e-ddb1f16cbf56 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.605113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.605807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.606312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.606624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] PATH_INFO: `/v3/projects/e81b2b8a45a44f37a8891d2e895be080/users/885286dd77f84777b4e3cfc4432ac843/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.622868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81b2b8a45a44f37a8891d2e895be080, user_id=885286dd77f84777b4e3cfc4432ac843, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.626478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.639763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1237/2477] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:56 2022] PUT /identity/v3/projects/e81b2b8a45a44f37a8891d2e895be080/users/885286dd77f84777b4e3cfc4432ac843/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:56.653862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.654149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.654425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.685096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] MFA Rules not processed for user `885286dd77f84777b4e3cfc4432ac843`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:56.686331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:56.729846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:56.761453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1241/2478] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:56.766593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.832163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bc78f-8b08-4812-a914-95912a7acaff admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.837475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.838081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.838365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.838641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.839278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.841238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.871428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1238/2479] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:56.875933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.887487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c107d98-d2de-4a56-8d5e-a99809c76fab None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.947056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.947758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.948102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.948453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.949163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.951218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:56.975103 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1242/2480] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:36:56 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:36:56.978752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:56.989663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d25b87-6d8b-40ce-adf3-9d8fad88eccc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:56.994360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:56.994984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:56.995262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:56.995525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:56.996183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:56.997851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.009805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1239/2481] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:36:57.014366 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:57.024947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabb15b-b30b-457a-9c1c-38ed5919c9be admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:57.029524 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:57.030175 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.030442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.030761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] PATH_INFO: `/v3/projects/84ba1f10789e4ba9954546da3c3518fd/users/fa03738435c140c49389d47a2c9b0cdd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.045840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ba1f10789e4ba9954546da3c3518fd, user_id=fa03738435c140c49389d47a2c9b0cdd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:57.048618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.061443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1243/2482] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:57 2022] PUT /identity/v3/projects/84ba1f10789e4ba9954546da3c3518fd/users/fa03738435c140c49389d47a2c9b0cdd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:57.066184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:57.076313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd0a7-e14b-47e8-be4f-586e6917bae4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:57.143058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:57.143658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.143929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.144195 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.144850 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:57.146454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d4613-65fd-48c5-9607-d4e521bd101f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.156588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1240/2483] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:36:57.160763 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:57.170884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff4b81-fda6-44cf-a18c-04fc3347f775 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:57.176094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:57.176768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.177047 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.177320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] PATH_INFO: `/v3/projects/84ba1f10789e4ba9954546da3c3518fd/users/fa03738435c140c49389d47a2c9b0cdd/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.179363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ba1f10789e4ba9954546da3c3518fd, user_id=fa03738435c140c49389d47a2c9b0cdd, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:57.182011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.192148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1244/2484] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:36:57 2022] PUT /identity/v3/projects/84ba1f10789e4ba9954546da3c3518fd/users/fa03738435c140c49389d47a2c9b0cdd/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:36:57.203183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.203470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.203744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.231297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] MFA Rules not processed for user `fa03738435c140c49389d47a2c9b0cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:57.232531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:57.272180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:57.301036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1241/2485] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:36:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:57.312345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:57.314072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:57.371531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62322dfd-74b1-42f0-b397-963538dc3ea1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:57.375273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:57.375860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.376134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.376435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.385153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:57.386551 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.396373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1245/2486] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:57.504981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:57.520924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd70d-339f-4e89-ab71-7c110edf8a34 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:57.525251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:57.525896 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.526052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.526324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.613750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:57.615385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181e25-4087-4f3d-9ada-377235d995ff service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:57.628197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1242/2487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:37:00.255124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:00.289283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:00.317988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01368d51-c8e2-41bf-a95d-ea847f88262d service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:00.322448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:00.323090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:00.323364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:00.323631 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] PATH_INFO: `/v3/users/885286dd77f84777b4e3cfc4432ac843` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:00.324760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=885286dd77f84777b4e3cfc4432ac843)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:00.326640 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:00.374563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885286dd77f84777b4e3cfc4432ac843'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.382743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885286dd77f84777b4e3cfc4432ac843'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.383587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885286dd77f84777b4e3cfc4432ac843'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.391236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885286dd77f84777b4e3cfc4432ac843'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.391859 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885286dd77f84777b4e3cfc4432ac843'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.408755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1246/2488] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:00 2022] DELETE /identity/v3/users/885286dd77f84777b4e3cfc4432ac843 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:00.413237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:00.413711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:00.413809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:00.433119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:00.434306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:00.493064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1243/2489] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:00.502933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:00.504379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:00.562517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40fb-14f3-4f3e-9523-1f24b63f1e9a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:00.566252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:00.566943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:00.567220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:00.567491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:00.576645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:00.578777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:00.588084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1247/2490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:00.852108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:00.854182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:00.866814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e13f2-c1e0-4129-acb1-25eb4aa2c320 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:00.871446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:00.872072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:00.872377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:00.872648 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] PATH_INFO: `/v3/projects/e81b2b8a45a44f37a8891d2e895be080` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:00.873915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81b2b8a45a44f37a8891d2e895be080)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:00.876452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:00.938253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81b2b8a45a44f37a8891d2e895be080', 'request_id': 'req-698f38ce-693c-4b61-a664-a336a5abf8ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.946234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81b2b8a45a44f37a8891d2e895be080', 'request_id': 'req-698f38ce-693c-4b61-a664-a336a5abf8ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.954208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81b2b8a45a44f37a8891d2e895be080', 'request_id': 'req-698f38ce-693c-4b61-a664-a336a5abf8ec'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.966942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] The token cache is being invalidate because project e81b2b8a45a44f37a8891d2e895be080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:00.972547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.973500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:00.974215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:00.976313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1244/2491] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:00 2022] DELETE /identity/v3/projects/e81b2b8a45a44f37a8891d2e895be080 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:00.980345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:00.990318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ac9b5-98f2-451c-89d4-11f19354cce5 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.047912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:01.048845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.050364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.061189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.062630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.062949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.063229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] PATH_INFO: `/v3/users/fa03738435c140c49389d47a2c9b0cdd` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.064472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa03738435c140c49389d47a2c9b0cdd)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.066656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.130323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa03738435c140c49389d47a2c9b0cdd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.142233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa03738435c140c49389d47a2c9b0cdd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.142906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa03738435c140c49389d47a2c9b0cdd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.156187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38ce-693c-4b61-a664-a336a5abf8ec admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.156766 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa03738435c140c49389d47a2c9b0cdd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.158071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa03738435c140c49389d47a2c9b0cdd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.167270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.173647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.173647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.173647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] PATH_INFO: `/v3/users/f2f12856ec9045ef804d6a54134e7f22` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.173647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f12856ec9045ef804d6a54134e7f22)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.173647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.200129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1248/2492] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:00 2022] DELETE /identity/v3/users/fa03738435c140c49389d47a2c9b0cdd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:01.201013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.202847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.232693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f12856ec9045ef804d6a54134e7f22'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.234286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f12856ec9045ef804d6a54134e7f22'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.242989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f12856ec9045ef804d6a54134e7f22'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.243777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f12856ec9045ef804d6a54134e7f22'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.255337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-37e45b7f-3c62-42fd-86c1-a713e5886658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f12856ec9045ef804d6a54134e7f22'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.273285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd52c-1e20-4bac-ab4f-e846fc16c399 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.277498 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.278551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1245/2493] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:01 2022] DELETE /identity/v3/users/f2f12856ec9045ef804d6a54134e7f22 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:01.278678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.278678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.279096 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.282916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.283250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.283472 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.284039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.304536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:01.305803 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.359077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.360877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.371692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1249/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:01.375399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1246/2495] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:01.417275 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.419245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.431125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75213864-07e5-4931-9d11-01f0d33d8162 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.452461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:01.492442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.493168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.493519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.493861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.503937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.505687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb169-0bfc-41d0-b934-030fc2822641 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.516595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.518379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.521470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1250/2496] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:37:01.534431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d32dc-ed12-4bf9-9871-74cc130f953b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.540041 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.541064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.541342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.541641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] PATH_INFO: `/v3/projects/84ba1f10789e4ba9954546da3c3518fd` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.542860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ba1f10789e4ba9954546da3c3518fd)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.544764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.560125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:01.599873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ba1f10789e4ba9954546da3c3518fd', 'request_id': 'req-6c96503d-44e4-42b3-84f2-8371976791aa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.609564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ba1f10789e4ba9954546da3c3518fd', 'request_id': 'req-6c96503d-44e4-42b3-84f2-8371976791aa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.616075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ba1f10789e4ba9954546da3c3518fd', 'request_id': 'req-6c96503d-44e4-42b3-84f2-8371976791aa'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.628337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] The token cache is being invalidate because project 84ba1f10789e4ba9954546da3c3518fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:01.628633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.629950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c96503d-44e4-42b3-84f2-8371976791aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:01.631840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1247/2497] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:01 2022] DELETE /identity/v3/projects/84ba1f10789e4ba9954546da3c3518fd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:01.636606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.636995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.637269 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.658705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:01.659911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.722279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1251/2498] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:01.730255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.730538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.730814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.752046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:01.753345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.761325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1248/2499] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:01.766221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.775816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef423f2-5e65-46fb-add6-cab1734766cb None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.780566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.781100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.781377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.781645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.782281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:01.782602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.784187 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.797342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1252/2500] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:37:01.801695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.811166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64acdb7-91f5-42dc-a903-cf5c182bf107 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.816383 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.817022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.817298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.817565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.818208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.820106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.849457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1249/2501] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:01.853890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.864922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef398621-dd6c-4f74-8037-ad8c6c5f1d29 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.925484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.926094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.926494 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.926771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.927413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.929380 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:01.956972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1253/2502] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:01.960971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.978478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b192-b037-490b-abbb-8e9bdb1519ad admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.984149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:01.984798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.985118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.985397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.986039 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:01.988471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.000148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1250/2503] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:02.004356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.016779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc60a8d-a656-4959-a6f0-f712f14c7cbb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.023743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.024500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.024845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.025189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.025911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.027666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.042664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1254/2504] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:02.047206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.056496 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.059294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.059970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85659a8-1d6f-4012-89b6-831012949a6f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.065617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.074304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8660-2429-4e8d-92f8-92646cf4e4da admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.078254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.078763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.086211 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.086958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.087223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.087756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] PATH_INFO: `/v3/projects/68d09451a7774b63bc312bf0de415e01` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.089147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d09451a7774b63bc312bf0de415e01)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.089871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] PATH_INFO: `/v3/projects/bd1767be02f5469fac353f6ac9e72fa4/users/0a98444d721541cbb73bcf2f4954f51a/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.091085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.123790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1767be02f5469fac353f6ac9e72fa4, user_id=0a98444d721541cbb73bcf2f4954f51a, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.127426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60ed0b-eae9-449c-a360-6c7b0799d1d3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.128499 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.146998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1251/2505] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:02 2022] PUT /identity/v3/projects/bd1767be02f5469fac353f6ac9e72fa4/users/0a98444d721541cbb73bcf2f4954f51a/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:02.147353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d09451a7774b63bc312bf0de415e01', 'request_id': 'req-c5e19d63-237b-4610-9772-b14cc96b786d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.155046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d09451a7774b63bc312bf0de415e01', 'request_id': 'req-c5e19d63-237b-4610-9772-b14cc96b786d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.158304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.158973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.158973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.164585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d09451a7774b63bc312bf0de415e01', 'request_id': 'req-c5e19d63-237b-4610-9772-b14cc96b786d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.176438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] The token cache is being invalidate because project 68d09451a7774b63bc312bf0de415e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:02.176743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.177401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c5e19d63-237b-4610-9772-b14cc96b786d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.179197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1255/2506] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:02 2022] DELETE /identity/v3/projects/68d09451a7774b63bc312bf0de415e01 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:02.183707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98afd-e512-44ee-8b50-a3c377366b96 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.184145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98afd-e512-44ee-8b50-a3c377366b96 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.184870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c98afd-e512-44ee-8b50-a3c377366b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.189664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] MFA Rules not processed for user `0a98444d721541cbb73bcf2f4954f51a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.190889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.207737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b8c98afd-e512-44ee-8b50-a3c377366b96 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.209056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b8c98afd-e512-44ee-8b50-a3c377366b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.270352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1252/2507] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:02.280906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1256/2508] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:02.285465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.286484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.286484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.286793 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.286917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.314767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.316287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.329226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1257/2509] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:02.330461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.335190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.341019 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205803b-de58-4781-ae03-426ace20c5b1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.348514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.349350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.349695 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.350031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.350740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:02.351645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.352668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.367410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1258/2510] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:37:02.371612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b46ec-7dee-4af4-9e9a-9e7ff383b655 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.371838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.375676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.376317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.376590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.376866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.382740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142c948-88dd-4945-8294-78235a822a37 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.387405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.387944 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.388776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.389122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.389476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.390131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.391148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.393353 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.398773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1253/2511] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:02.421627 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.423410 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.432594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1259/2512] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:02.434414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.438625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e1426-5902-40f2-918d-cfcf453c490a service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.507583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.508223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.508558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.508831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] PATH_INFO: `/v3/users/0a98444d721541cbb73bcf2f4954f51a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.509977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a98444d721541cbb73bcf2f4954f51a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.510700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f01460-3426-4348-a77a-fe011fc36db3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.511901 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.517164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.517773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.518057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.519054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.519054 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.520900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.545149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1260/2513] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:02.550379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.557440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.563513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d92686-a5ac-439f-af21-cc792a8006ae admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.569717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.570335 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.570619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.570904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.571774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.573910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.577840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a98444d721541cbb73bcf2f4954f51a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.578778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a98444d721541cbb73bcf2f4954f51a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.586101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1261/2514] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:02.589022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a98444d721541cbb73bcf2f4954f51a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.590650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a98444d721541cbb73bcf2f4954f51a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.597362 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.599601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.606978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-98d8886e-7f3c-4e4b-8f3e-711508d6cfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a98444d721541cbb73bcf2f4954f51a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:02.619926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe3a8-cb0b-44d7-b6a1-f9c4faac9105 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.625969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.626574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.626846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.627118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.627764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.629407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.635768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1254/2515] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:02 2022] DELETE /identity/v3/users/0a98444d721541cbb73bcf2f4954f51a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:02.637644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.638030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.638538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.650162 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1262/2516] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:02.654088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.670570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.672096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.672891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732dae4-e01f-4ca6-acd7-403748e8a7cc admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.727481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.749834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1255/2517] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:02.752556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.753617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.753898 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.754169 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] PATH_INFO: `/v3/projects/f88ccc88ce1f40f9a250800f256cb0ed/users/af9f698509074a6db2d5342cbc4946bf/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.760333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.762619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.771632 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88ccc88ce1f40f9a250800f256cb0ed, user_id=af9f698509074a6db2d5342cbc4946bf, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.774612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc68fb2-820b-4155-ae2c-c93f1daa32dc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.790809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1263/2518] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:02 2022] PUT /identity/v3/projects/f88ccc88ce1f40f9a250800f256cb0ed/users/af9f698509074a6db2d5342cbc4946bf/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:02.807952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.808600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.809244 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.843748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] MFA Rules not processed for user `af9f698509074a6db2d5342cbc4946bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.845645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.850024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22ae47-9769-4e3f-a4ed-b5c40ee28386 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.918416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.919064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.919266 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.919512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.923174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1264/2519] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:02.934581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.937342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.008634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.010354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.016713 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5817b-b887-47a6-896a-ca0f08e81914 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.020648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.021292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.021629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.021909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.024010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1256/2520] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:03.031440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.033041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dee20-d680-4c7c-98e5-c99f63935c03 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.036710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:03.044214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1265/2521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:37:03.071192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.090240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d845f-fcdc-4987-9708-e186064da6c3 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.095586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.096484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.096856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.097134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] PATH_INFO: `/v3/users/af9f698509074a6db2d5342cbc4946bf` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.098666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af9f698509074a6db2d5342cbc4946bf)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.101097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.169426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af9f698509074a6db2d5342cbc4946bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.170559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af9f698509074a6db2d5342cbc4946bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.175833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:03.180797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af9f698509074a6db2d5342cbc4946bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.182903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af9f698509074a6db2d5342cbc4946bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.197221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af9f698509074a6db2d5342cbc4946bf'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.221021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1257/2522] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:03 2022] DELETE /identity/v3/users/af9f698509074a6db2d5342cbc4946bf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:03.226176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.226461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.226735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.251411 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:03.252864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.347858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1266/2523] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:03.365552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.367102 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.419446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.421689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.443161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a8860-fc6e-472f-88e7-8818d68f8af6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.448198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.448875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.449151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.449471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] PATH_INFO: `/v3/projects/bd1767be02f5469fac353f6ac9e72fa4` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.451912 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1767be02f5469fac353f6ac9e72fa4)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.453946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.475131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1832e-9e15-4a3f-bfa1-20415fefac1b admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.480882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.481509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.482324 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.482612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.500310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.501913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42f563-c5c0-4254-90d9-4ee10f6ef6fe service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.509579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1767be02f5469fac353f6ac9e72fa4', 'request_id': 'req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.516975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1258/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:03.517305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1767be02f5469fac353f6ac9e72fa4', 'request_id': 'req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.525989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1767be02f5469fac353f6ac9e72fa4', 'request_id': 'req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.539650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] The token cache is being invalidate because project bd1767be02f5469fac353f6ac9e72fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:03.540148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.541021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2b65f94c-44ae-4c64-b2c9-e8a69b6a8da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:03.543068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1267/2525] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:03 2022] DELETE /identity/v3/projects/bd1767be02f5469fac353f6ac9e72fa4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:03.548832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.549125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.549403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.574146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:03.575741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.659604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1259/2526] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:03.668415 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.669323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.669323 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.692714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:03.694371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.706822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1268/2527] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:03.708032 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.720991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d5f94-d6cf-49b9-9ac5-1c7538a74a74 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.726513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.727128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.727401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.727673 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.728344 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:03.728644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.730630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.744988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1260/2528] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:37:03.749655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.761629 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38929473-4ec0-4397-ae6e-f267698611cf None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.767572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.767952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.768367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.768666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.769303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.771199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.808907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1269/2529] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:03.814771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.829767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4ae48-4070-4a90-92d8-f664107797bc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.861210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.863255 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.917998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:03.926914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.928089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.928676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.928955 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.929585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.931524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.937900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:03.959964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1261/2530] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:03.961271 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:03.964044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999db6-e18d-48ca-8bda-0d56b546b658 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.971474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.971816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec1dde-d3e1-4a41-b359-0c41d72044e2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:03.972553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.972835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.973513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] PATH_INFO: `/v3/projects/f88ccc88ce1f40f9a250800f256cb0ed` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.976587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88ccc88ce1f40f9a250800f256cb0ed)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.977788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:03.978396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.978682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.978983 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:03.979480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.979640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.981306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.997912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1262/2531] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:04.001242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.011495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91509e-d4eb-4d75-a8d7-c6ff0e1b2a72 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.017859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.018488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.019127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.019127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.019651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.021313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.032199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1263/2532] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:04.036597 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.040660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88ccc88ce1f40f9a250800f256cb0ed', 'request_id': 'req-5d0d4156-7f5d-4486-be81-3e80395cb0fa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:04.048736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0ca6-1d47-44d5-ba6e-7808cabf01b3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.048905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88ccc88ce1f40f9a250800f256cb0ed', 'request_id': 'req-5d0d4156-7f5d-4486-be81-3e80395cb0fa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:04.057249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88ccc88ce1f40f9a250800f256cb0ed', 'request_id': 'req-5d0d4156-7f5d-4486-be81-3e80395cb0fa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:04.071478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] The token cache is being invalidate because project f88ccc88ce1f40f9a250800f256cb0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:04.072147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:04.072897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-5d0d4156-7f5d-4486-be81-3e80395cb0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:04.074742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1270/2533] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:03 2022] DELETE /identity/v3/projects/f88ccc88ce1f40f9a250800f256cb0ed => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:04.083540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d305-dc5f-4137-a4e3-4ab6d11e2c4b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.083540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d305-dc5f-4137-a4e3-4ab6d11e2c4b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.083540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d305-dc5f-4137-a4e3-4ab6d11e2c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.102113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-f787d305-dc5f-4137-a4e3-4ab6d11e2c4b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.103433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-f787d305-dc5f-4137-a4e3-4ab6d11e2c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.127587 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.128399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.128727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.129012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] PATH_INFO: `/v3/projects/b30bfa67be7e44359cd5f3a9d46e8a24/users/f59d07d19ded4f17afb4146def3f924d/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.155518 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30bfa67be7e44359cd5f3a9d46e8a24, user_id=f59d07d19ded4f17afb4146def3f924d, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.163561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bd53d-600c-4566-bb39-de28f78dd602 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.186425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1264/2534] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:04 2022] PUT /identity/v3/projects/b30bfa67be7e44359cd5f3a9d46e8a24/users/f59d07d19ded4f17afb4146def3f924d/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:04.197421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.197831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.198212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.221345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1271/2535] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.223817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.224207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.224607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.236347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] MFA Rules not processed for user `f59d07d19ded4f17afb4146def3f924d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.237230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.247291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.248556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.324572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1272/2536] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.330070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.344615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a216a5-457a-4872-80ce-384ba873c459 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.351183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1265/2537] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:04.354437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.355190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.355538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.355882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.356512 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.358118 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.362767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:04.365008 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.367036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.375334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afc2e-c43c-478d-880d-dc5a5950110a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.382537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.383163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.383440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.383713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.384385 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.385018 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1273/2538] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:37:04.388495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.391110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.402221 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc17701-da1c-46a5-8c24-5fdd86fa8721 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.408611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.409456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.409882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.410906 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.411743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.413759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.418979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1266/2539] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:04.422975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.426622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.443727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.449780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1274/2540] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:04.454322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.502515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336eb2-c908-4491-b017-2792fcd5f67e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.528092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad976636-ba77-4bec-a1b0-4ab873994293 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.531258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.531979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.532357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.532699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.533410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.535397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.563480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1275/2541] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:04.568002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.577976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.578661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.578950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.579374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.580010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.580754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e7211-2cff-467e-83af-c0df6a6d258b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.582306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.585438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.586129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.586517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.586834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.587651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.589530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.605780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1276/2542] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:04.606824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.616393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1267/2543] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:04.618639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.618773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89d649-f076-4809-80aa-ff64fe529659 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.625434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.626213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.626520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.626947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.627742 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.629322 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.644052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1277/2544] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:04.648733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.651804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df91d7e-2223-4f19-8ac6-6474330a34f9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.661804 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.663488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.663939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.664474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.665278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.667355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.670564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de48d7-a068-4051-bb81-5afb7b4adad3 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.679032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.679724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.680006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.680328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] PATH_INFO: `/v3/projects/acb97fc9f6ca4cc3970ba2192c2fa3c1/users/12f9f5e7d546480ea24913421273d9b4/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.688523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1268/2545] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:04.696375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.706734 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c41b2-8c9c-4f98-936a-96f1b8a66daa admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.711733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.712546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.712893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.713230 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.714148 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.715802 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecc4b6-ff92-4e1e-a196-4c7b2ccfc531 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.722413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb97fc9f6ca4cc3970ba2192c2fa3c1, user_id=12f9f5e7d546480ea24913421273d9b4, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.726166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.742469 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1269/2546] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:04.743363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1278/2547] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:04 2022] PUT /identity/v3/projects/acb97fc9f6ca4cc3970ba2192c2fa3c1/users/12f9f5e7d546480ea24913421273d9b4/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:04.748527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.748717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.749558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.750257 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.773045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.774892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.833112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27911-99d7-4319-a7e7-0b7a0a35179b admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.839154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.840157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.840661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.841092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] PATH_INFO: `/v3/projects/116a96c8cce245678fd0a531b0a5b363/users/ae77470d65824ec9a7657bbe3de92823/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.862378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1270/2548] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.864583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.867635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.869126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116a96c8cce245678fd0a531b0a5b363, user_id=ae77470d65824ec9a7657bbe3de92823, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.873683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213e8b1-41f6-43ac-b259-9be9d623eac5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.901276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1279/2549] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:04 2022] PUT /identity/v3/projects/116a96c8cce245678fd0a531b0a5b363/users/ae77470d65824ec9a7657bbe3de92823/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:04.912994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.916073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.916699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.917148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.944545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f52ef-71fa-4c9c-bec0-149f71d02dd7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.978656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] MFA Rules not processed for user `ae77470d65824ec9a7657bbe3de92823`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.980070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.986487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:05.026726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.027514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.027795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.028205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.084070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1280/2550] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:05.089556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.142163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.143310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.156167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1271/2551] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:05.187872 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d58e7-ebc4-4b0f-a0a9-5ccf93493897 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.196354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.197397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.197779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.198172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.198905 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.201385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.243590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1281/2552] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:05.245759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.260954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a98819-8986-457e-9080-d01ab69b0012 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.369180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.369810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.370228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.370508 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.371669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.374500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.399814 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1272/2553] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:05 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:05.408845 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.423608 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28727f3-def6-45d6-823c-2db2ba216cc0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.429117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.429812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.430170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.430619 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.431239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.432950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.456349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1282/2554] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:05.457789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.469396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05906bd-d9cf-4ed6-a168-03b0c37cffef admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.475062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.475746 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.476027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.476423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] PATH_INFO: `/v3/projects/2ce1abcb5dce4e89904e387df3eb8373/users/6e7e927eb7144909bb2043c13bf0bba7/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.493523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce1abcb5dce4e89904e387df3eb8373, user_id=6e7e927eb7144909bb2043c13bf0bba7, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.496358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.512987 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1273/2555] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:05 2022] PUT /identity/v3/projects/2ce1abcb5dce4e89904e387df3eb8373/users/6e7e927eb7144909bb2043c13bf0bba7/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:05.518900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.532406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224880b-6d88-4135-bb1f-016a93b3e82d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.640705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.641714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.642178 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.642630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.643594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.645475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a98a-743d-4fc4-b90c-b352831c9690 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.660320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1283/2556] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:05.664710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.671836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:05.679879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019c78c-db66-4979-90af-e8f29734a9d1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.687946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.688721 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.689248 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.689525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] PATH_INFO: `/v3/projects/2ce1abcb5dce4e89904e387df3eb8373/users/6e7e927eb7144909bb2043c13bf0bba7/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.691615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce1abcb5dce4e89904e387df3eb8373, user_id=6e7e927eb7144909bb2043c13bf0bba7, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.695937 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.731304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1274/2557] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:05 2022] PUT /identity/v3/projects/2ce1abcb5dce4e89904e387df3eb8373/users/6e7e927eb7144909bb2043c13bf0bba7/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:05.781962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.782267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.782531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.812907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] MFA Rules not processed for user `6e7e927eb7144909bb2043c13bf0bba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:05.814150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:05.818295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:05.909295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1284/2558] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:05.922120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:05.924731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:06.000176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8040a9-6abb-431f-9f5c-f40dbf34a793 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:06.004862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:06.005834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:06.006265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:06.006726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:06.094869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:06.096552 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:06.107617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1275/2559] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:07.973616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:07.977098 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:07.990429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57345326-5160-4b2d-92b1-c6003c288ca2 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:07.994570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:07.995228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:07.995660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:07.995936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:08.074203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:08.076160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:08.100271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1285/2560] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:09.026638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:09.054291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbccc38-74b0-4f9c-a4f6-ef7e42440ee4 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:09.068905 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:09.069698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:09.069988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:09.070263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:09.130709 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:09.134286 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef27cb-f17b-439a-8ef6-4f32653805f6 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:09.161750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1276/2561] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:10.324617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:10.326128 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:10.547582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:10.584072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757f2f4-fe71-46f2-8312-e735ffdfcaf7 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:10.593964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:10.595328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:10.595938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:10.596427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] PATH_INFO: `/v3/users/a0794e19f43a413db886750b0517d3d3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:10.598562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0794e19f43a413db886750b0517d3d3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:10.600555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:10.711821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0794e19f43a413db886750b0517d3d3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:10.724479 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0794e19f43a413db886750b0517d3d3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:10.725544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0794e19f43a413db886750b0517d3d3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:10.746768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0794e19f43a413db886750b0517d3d3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:10.747575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0794e19f43a413db886750b0517d3d3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:10.819223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1286/2562] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:10 2022] DELETE /identity/v3/users/a0794e19f43a413db886750b0517d3d3 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:11.095743 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.096269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.096374 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.146699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] MFA Rules not processed for user `12f9f5e7d546480ea24913421273d9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:11.148533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.165924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.168028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.227141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:11.259114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1277/2563] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:11.263910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.264724 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274560-f2af-47e5-ab1e-0e11d8f2209f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.265592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.271081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.271712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.271993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.272327 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] PATH_INFO: `/v3/projects/e1b5f9daf8d44a6987bbff8939ed7c92` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.273417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b5f9daf8d44a6987bbff8939ed7c92)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.275309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.280966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c0cd-df06-4f56-acd7-4dada92df58d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.285820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.286449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.286722 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.286988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.287614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.289534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.320071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1278/2564] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:11 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:11.320982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.326429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b5f9daf8d44a6987bbff8939ed7c92', 'request_id': 'req-3f66ffd8-c5fb-48d8-851f-260b40911c78'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.333236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b5f9daf8d44a6987bbff8939ed7c92', 'request_id': 'req-3f66ffd8-c5fb-48d8-851f-260b40911c78'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.343334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b5f9daf8d44a6987bbff8939ed7c92', 'request_id': 'req-3f66ffd8-c5fb-48d8-851f-260b40911c78'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.349207 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:11.357265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] The token cache is being invalidate because project e1b5f9daf8d44a6987bbff8939ed7c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:11.357730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.359089 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.363858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1287/2565] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:11 2022] DELETE /identity/v3/projects/e1b5f9daf8d44a6987bbff8939ed7c92 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:11.365472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.367289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.388014 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416b2f-cae7-49c1-9932-330fb5b4cd93 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.393550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.394146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.394420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.394685 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.395452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.397467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.422033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1279/2566] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:11 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:11.425935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.427554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.436229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66ffd8-c5fb-48d8-851f-260b40911c78 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.439886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffe95-ca2d-4927-a923-f7e7fdf38f31 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.441046 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.441838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.442180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.442371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] PATH_INFO: `/v3/users/5fb81b0d881544c6a16fc19dd9ef9a55` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.443493 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb81b0d881544c6a16fc19dd9ef9a55)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.444570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.445158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.445435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.445561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.445736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.446332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.447910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.457872 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1280/2567] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:11.462702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.474162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b57e59-0b26-4573-8787-f386b68306f0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.479973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.480911 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.481187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.481503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] PATH_INFO: `/v3/projects/16f4765a996048b4930ed42949a4cb58/users/94a60b46842f41c1852ea86c196701ca/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.493955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb81b0d881544c6a16fc19dd9ef9a55'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.498761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f4765a996048b4930ed42949a4cb58, user_id=94a60b46842f41c1852ea86c196701ca, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.501117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb81b0d881544c6a16fc19dd9ef9a55'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.501745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb81b0d881544c6a16fc19dd9ef9a55'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.501923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.509129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb81b0d881544c6a16fc19dd9ef9a55'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.509910 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb81b0d881544c6a16fc19dd9ef9a55'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.514653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1281/2568] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:11 2022] PUT /identity/v3/projects/16f4765a996048b4930ed42949a4cb58/users/94a60b46842f41c1852ea86c196701ca/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:11.518678 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.520378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.529587 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1288/2569] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:11 2022] DELETE /identity/v3/users/5fb81b0d881544c6a16fc19dd9ef9a55 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:11.588279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c0306-3f6f-46a3-b4cf-40af3852c14e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.616991 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:11.651233 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.651959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.652355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.652604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.653527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.656026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.666304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1282/2570] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:11.670603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.680641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe17e-c0bd-48d1-a664-852e8b875d0e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.685397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.686084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.686361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.686625 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] PATH_INFO: `/v3/projects/16f4765a996048b4930ed42949a4cb58/users/94a60b46842f41c1852ea86c196701ca/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.688694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f4765a996048b4930ed42949a4cb58, user_id=94a60b46842f41c1852ea86c196701ca, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.691334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.707135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1289/2571] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:11 2022] PUT /identity/v3/projects/16f4765a996048b4930ed42949a4cb58/users/94a60b46842f41c1852ea86c196701ca/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:11.729782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:11.732089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:11.753764 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ced0a0-49ae-414e-bc11-e265dd6c028d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:11.850989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:11.851666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:11.851949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:11.852231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] PATH_INFO: `/v3/projects/87f5261f400a414d9b28fc8ce213b317` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:11.853437 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f5261f400a414d9b28fc8ce213b317)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:11.855336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:11.919752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f5261f400a414d9b28fc8ce213b317', 'request_id': 'req-0afbb758-3e2f-4331-a73e-fc280fceb599'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.940760 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f5261f400a414d9b28fc8ce213b317', 'request_id': 'req-0afbb758-3e2f-4331-a73e-fc280fceb599'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.958149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f5261f400a414d9b28fc8ce213b317', 'request_id': 'req-0afbb758-3e2f-4331-a73e-fc280fceb599'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.977511 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] The token cache is being invalidate because project 87f5261f400a414d9b28fc8ce213b317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:11.977811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.978546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0afbb758-3e2f-4331-a73e-fc280fceb599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:11.980299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1283/2572] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:11 2022] DELETE /identity/v3/projects/87f5261f400a414d9b28fc8ce213b317 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:13.645618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:13.648540 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:13.667143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743be65a-984d-4732-bb16-bb4a0323d793 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:13.739693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:13.740750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:13.741198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:13.741630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] PATH_INFO: `/v3/users/f59d07d19ded4f17afb4146def3f924d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:13.743052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59d07d19ded4f17afb4146def3f924d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:13.745302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:13.747382 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:13.810531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59d07d19ded4f17afb4146def3f924d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:13.832933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59d07d19ded4f17afb4146def3f924d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:13.834136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59d07d19ded4f17afb4146def3f924d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:13.850977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59d07d19ded4f17afb4146def3f924d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:13.851831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59d07d19ded4f17afb4146def3f924d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:13.875351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1290/2573] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:13 2022] DELETE /identity/v3/users/f59d07d19ded4f17afb4146def3f924d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:13.879823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:13.880358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:13.880474 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:13.910054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:13.911253 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:13.966712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:13.988784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1284/2574] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:14.001028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:14.002480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:14.081726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2f980-6a75-48a7-abc3-c5c7999e87bb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:14.088107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:14.088791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:14.089102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:14.089381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:14.100588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:14.102566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:14.119346 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1291/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:14.504940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:14.506565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:14.532745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea70206-f73f-402e-a9b2-e75fda3fe731 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:14.552051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:14.552733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:14.553159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:14.553279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] PATH_INFO: `/v3/projects/b30bfa67be7e44359cd5f3a9d46e8a24` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:14.556419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30bfa67be7e44359cd5f3a9d46e8a24)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:14.559105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:14.667838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30bfa67be7e44359cd5f3a9d46e8a24', 'request_id': 'req-82f0280a-4066-4bd1-af07-d3668915ed25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:14.695070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30bfa67be7e44359cd5f3a9d46e8a24', 'request_id': 'req-82f0280a-4066-4bd1-af07-d3668915ed25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:14.710493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30bfa67be7e44359cd5f3a9d46e8a24', 'request_id': 'req-82f0280a-4066-4bd1-af07-d3668915ed25'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:14.736548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] The token cache is being invalidate because project b30bfa67be7e44359cd5f3a9d46e8a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:14.736548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:14.737185 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:14.750390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1285/2576] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:14 2022] DELETE /identity/v3/projects/b30bfa67be7e44359cd5f3a9d46e8a24 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:14.750661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:14.782037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331acf0-544d-40e5-9a51-b46b9b3c84e5 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:14.861846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:14.863210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:14.863699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:14.864061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] PATH_INFO: `/v3/users/ae77470d65824ec9a7657bbe3de92823` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:14.865618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae77470d65824ec9a7657bbe3de92823)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:14.876157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:14.944617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:15.057658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae77470d65824ec9a7657bbe3de92823'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:15.069871 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae77470d65824ec9a7657bbe3de92823'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:15.070571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae77470d65824ec9a7657bbe3de92823'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:15.093570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae77470d65824ec9a7657bbe3de92823'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:15.094329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae77470d65824ec9a7657bbe3de92823'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:15.143409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1292/2577] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:14 2022] DELETE /identity/v3/users/ae77470d65824ec9a7657bbe3de92823 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:16.860369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:16.862413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:16.897200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0280a-4066-4bd1-af07-d3668915ed25 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:17.038616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:17.039433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:17.041171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:17.041553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] PATH_INFO: `/v3/projects/116a96c8cce245678fd0a531b0a5b363` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:17.045593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] RBAC: Authorizing `identity:delete_project(project_id=116a96c8cce245678fd0a531b0a5b363)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:17.050523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:17.181340 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116a96c8cce245678fd0a531b0a5b363', 'request_id': 'req-6ca33318-6235-4dca-ac91-856641fd848e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.205864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116a96c8cce245678fd0a531b0a5b363', 'request_id': 'req-6ca33318-6235-4dca-ac91-856641fd848e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.242924 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116a96c8cce245678fd0a531b0a5b363', 'request_id': 'req-6ca33318-6235-4dca-ac91-856641fd848e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.260332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] The token cache is being invalidate because project 116a96c8cce245678fd0a531b0a5b363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:17.261150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.262088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.265130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1286/2578] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:16 2022] DELETE /identity/v3/projects/116a96c8cce245678fd0a531b0a5b363 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:17.268914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:17.270582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:17.299464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b58fca-a983-45a1-9c70-7b200bb2ba1c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:17.450598 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:17.451347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:17.451616 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:17.451885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] PATH_INFO: `/v3/users/6e7e927eb7144909bb2043c13bf0bba7` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:17.461913 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7e927eb7144909bb2043c13bf0bba7)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:17.464149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:17.624986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7e927eb7144909bb2043c13bf0bba7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.644371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7e927eb7144909bb2043c13bf0bba7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.647288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7e927eb7144909bb2043c13bf0bba7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.660939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7e927eb7144909bb2043c13bf0bba7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.662079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7e927eb7144909bb2043c13bf0bba7'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:17.706351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1293/2579] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:17 2022] DELETE /identity/v3/users/6e7e927eb7144909bb2043c13bf0bba7 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:18.397677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:18.411226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:18.437235 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:18.444752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33318-6235-4dca-ac91-856641fd848e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:18.627153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:18.627811 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:18.628091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:18.628380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] PATH_INFO: `/v3/projects/2ce1abcb5dce4e89904e387df3eb8373` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:18.638076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce1abcb5dce4e89904e387df3eb8373)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:18.639320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:18.640690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:18.642387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:18.681804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:18.786954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce1abcb5dce4e89904e387df3eb8373', 'request_id': 'req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:18.802810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce1abcb5dce4e89904e387df3eb8373', 'request_id': 'req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:18.810331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce1abcb5dce4e89904e387df3eb8373', 'request_id': 'req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:18.824226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] The token cache is being invalidate because project 2ce1abcb5dce4e89904e387df3eb8373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:18.824657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:18.825182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7acb29d0-4581-4d15-bd0d-0e3409ec9ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:18.831388 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1287/2580] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:18 2022] DELETE /identity/v3/projects/2ce1abcb5dce4e89904e387df3eb8373 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:18.833492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f482d-8bc1-45fb-b26b-a2cb84862718 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:18.833887 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f482d-8bc1-45fb-b26b-a2cb84862718 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:18.834262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f482d-8bc1-45fb-b26b-a2cb84862718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:18.913673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1952-70c3-42d5-b5ee-3184812b2b08 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:18.937231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-fb5f482d-8bc1-45fb-b26b-a2cb84862718 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:18.938574 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fb5f482d-8bc1-45fb-b26b-a2cb84862718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:19.057569 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1288/2581] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:19.065910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.066624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.067111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.075677 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.076447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.076735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.077084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.089179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:19.106022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:19.109794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:19.125409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.131029 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.149063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1289/2582] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:19.151970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.156621 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1294/2583] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:19.163831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:19.176535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9997b-086b-4095-a2e0-64461733ea97 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.192394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.194475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.194754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.197001 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.197701 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:19.199590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.201355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.222104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1290/2584] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:37:19.225603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.251021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33c5a4-0e34-4ebf-823a-eeed344537e3 service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.271550 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.272413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.272796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.273167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.274611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.277291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.311916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1295/2585] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:19 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:19.316218 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.328101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba96c87-a0f6-42a7-aa24-b1307b233d94 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.395672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.396989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.396989 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.397364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.398066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.400585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.429532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1291/2586] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:19 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:19.430875 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.443950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded57b-4f1d-44a8-b3a8-ff69e80d9fd9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.454900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.455548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.455827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.456100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.457007 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.459748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.473709 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1296/2587] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:19.478690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.495940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485d82-4eae-44a1-a044-281b2cf26b83 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.503323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.504000 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.504326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.504578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.506024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.507700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbad75e-d3a7-44c5-aa93-c945c9081c55 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.530154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1292/2588] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:19.531407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.546628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2687a8c-8fb9-4261-ad99-2fc4c3c21610 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:19.558862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:19.559767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.560133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.560972 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] PATH_INFO: `/v3/projects/5d9006a30c084e3cb22610b649cb5ed2/users/27a4761813fc45dd88de2e264e8022d5/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.586685 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9006a30c084e3cb22610b649cb5ed2, user_id=27a4761813fc45dd88de2e264e8022d5, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:19.590630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:19.609978 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1297/2589] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:19 2022] PUT /identity/v3/projects/5d9006a30c084e3cb22610b649cb5ed2/users/27a4761813fc45dd88de2e264e8022d5/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:19.621812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ee-333a-4d23-8653-4801538f6f8c None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:19.622188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ee-333a-4d23-8653-4801538f6f8c None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:19.622536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25101ee-333a-4d23-8653-4801538f6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:19.679180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a25101ee-333a-4d23-8653-4801538f6f8c None None] MFA Rules not processed for user `27a4761813fc45dd88de2e264e8022d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:19.680942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a25101ee-333a-4d23-8653-4801538f6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:19.903497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1293/2590] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:19.921719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:19.972958 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:20.038620 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7f398-6d1a-4827-bef4-c71cea829d6c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.044020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.045454 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.046132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.046545 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.047535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.054191 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.086581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.089149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.089457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.106894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1298/2591] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:19 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:20.111986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.146680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] MFA Rules not processed for user `94a60b46842f41c1852ea86c196701ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:20.148078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.213071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70540efb-750c-460e-8353-19e08211f95a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.219656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.220387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.220673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.220947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.221637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.224006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.236494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:20.253736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1299/2592] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:20.266576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.288100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1294/2593] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:20.299211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.300006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9733-667b-4e2a-9dcf-1c53e7b20b5f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.302208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.306225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.307235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.307687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.308232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.309144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.312402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.328149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1300/2594] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:20.332579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.346892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb755e-e377-4ffb-8d69-71404e7f7144 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.354064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.354885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.355159 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.355467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.356228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.358745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73a70e-2f15-4a34-b08c-446ae017a807 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.374816 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1301/2595] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:20.377040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.377359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.377745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.418384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] MFA Rules not processed for user `57daecc890ac421b8eb7b5fb4bf25411`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:20.421581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.440671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e17d38-9dc6-4f95-8e43-3730b63dfdaf None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.446503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.447243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.447527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.447798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.449605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.495840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1302/2596] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:20.497390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.513384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e9298-84d7-4f71-88b6-eea2a52f170f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.521636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.523011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.523011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.523235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] PATH_INFO: `/v3/projects/36414b20dc9745bf80f6fa0692b3f656/users/28fd7c9270664829ba2efab602e6f2b3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.550505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36414b20dc9745bf80f6fa0692b3f656, user_id=28fd7c9270664829ba2efab602e6f2b3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.553277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e931f65-d4c9-4364-9600-e7a4b3ae1e76 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.563359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:20.572104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1303/2597] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:20 2022] PUT /identity/v3/projects/36414b20dc9745bf80f6fa0692b3f656/users/28fd7c9270664829ba2efab602e6f2b3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:20.584291 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455ed9-405b-4f21-9088-72bfe16fc913 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.584584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455ed9-405b-4f21-9088-72bfe16fc913 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.584868 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455ed9-405b-4f21-9088-72bfe16fc913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.593034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.594489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.611824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1295/2598] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:20.666879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-68455ed9-405b-4f21-9088-72bfe16fc913 None None] MFA Rules not processed for user `28fd7c9270664829ba2efab602e6f2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:20.668212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-68455ed9-405b-4f21-9088-72bfe16fc913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.817987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1304/2599] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:20.822766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:20.837789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9c53f-575f-4ce9-b51e-678a2f7a4ef5 service glance] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:20.840320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.840672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.841023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.888164 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:20.890459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:20.926852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:20.927900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:20.928413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:20.928890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:20.930377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:20.933391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:20.994371 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1296/2600] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:20.995065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.021769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1305/2601] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:21.040947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.042508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:21.048536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:21.099473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abaa2a-3342-4081-9050-ff26d07fdc27 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.110906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.111588 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.111882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.112470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.113899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.115596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.144956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1297/2602] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:20 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:21.148801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.161025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6927-c5dd-4ddf-a8e6-3f64436fd6fb admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.162759 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b152ae9-13b8-4306-9520-0e70e3e15815 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.166158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.166994 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.167717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.167717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.168206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.170141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.183350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1298/2603] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:37:21.184390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:21.187069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.200173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f1980-e4d2-4b39-aa28-557455620d8e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.214190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.214870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.215150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.215430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] PATH_INFO: `/v3/projects/51847fb43ee2475dabec6d0d968b1d67/users/36842815057a4fee93d752494d881546/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.236914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51847fb43ee2475dabec6d0d968b1d67, user_id=36842815057a4fee93d752494d881546, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.240002 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.256182 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.256791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1299/2604] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:21 2022] PUT /identity/v3/projects/51847fb43ee2475dabec6d0d968b1d67/users/36842815057a4fee93d752494d881546/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:21.256976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.257350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.257637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.260435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.333662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.335477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.345205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1306/2605] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:37:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:37:21.346083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.348220 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:21.351825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe955-8e4d-4fb5-8e34-16e417cf87b7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.357065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.358341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.358687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.359022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.359735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.361589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.377605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1300/2606] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:21.378586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.392376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067e7e3-ee6c-4556-b514-a0be19af0274 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.399938 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.400658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.401004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.401441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] PATH_INFO: `/v3/projects/51847fb43ee2475dabec6d0d968b1d67/users/36842815057a4fee93d752494d881546/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.406219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51847fb43ee2475dabec6d0d968b1d67, user_id=36842815057a4fee93d752494d881546, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.408695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e9e99-9918-4935-93be-9a3dde6b26e1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.428069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1301/2607] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:21 2022] PUT /identity/v3/projects/51847fb43ee2475dabec6d0d968b1d67/users/36842815057a4fee93d752494d881546/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:37:21.439100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.439532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.440406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.444897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa686-b7ae-4d8e-afac-fa7a471e6d4f service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.499468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] MFA Rules not processed for user `36842815057a4fee93d752494d881546`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:21.507104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:21.575130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.575754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.576073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.577203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.579981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:21.629174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:21.639694 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:21.707455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1302/2608] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:21.719655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.719866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:21.721312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:21.721440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.741277 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1307/2609] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:21.805993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0cf95-7885-4f15-8320-116a60dec493 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:21.810252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:21.811498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:21.811836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:21.812117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:21.890926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:21.930079 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:21.930828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1303/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:24.193867 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:24.215097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b38bc-6f35-4362-884e-30dcacab458c service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:24.228151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:24.228804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:24.229085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:24.229364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:24.353233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:24.354680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:24.368276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1308/2611] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:25.825500 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:25.829421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:25.885819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6b855-3522-4fd1-8c2c-b61c235df732 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:25.910774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:25.912676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:25.916596 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:25.920094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:25.962693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:25.968040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:26.052306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1304/2612] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:29.207761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:29.209302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:29.242085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa3c9-dc95-4106-a7bf-94436d56eacf service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:29.257150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:29.257578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:29.257854 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:29.258129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] PATH_INFO: `/v3/users/a8f59b47841a4b62b8e368562c7bd210` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:29.261497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f59b47841a4b62b8e368562c7bd210)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:29.263559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:29.353838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f59b47841a4b62b8e368562c7bd210'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.370113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f59b47841a4b62b8e368562c7bd210'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.371130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f59b47841a4b62b8e368562c7bd210'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.383962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f59b47841a4b62b8e368562c7bd210'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.384679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-e472fd12-7779-41fd-af98-844e591318a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f59b47841a4b62b8e368562c7bd210'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.440423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1309/2613] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:29 2022] DELETE /identity/v3/users/a8f59b47841a4b62b8e368562c7bd210 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:29.725070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:29.742433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d356725-de32-44cd-a0f4-bee72ac641ef service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:29.874116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:29.874906 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:29.875082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:29.875273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] PATH_INFO: `/v3/projects/e5e1c15ae90e4b82b4e216b3a0549e2e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:29.878484 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e1c15ae90e4b82b4e216b3a0549e2e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:29.880433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:29.945653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:29.973892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e1c15ae90e4b82b4e216b3a0549e2e', 'request_id': 'req-42277c9f-33cd-4d7d-8651-d0b9c1b63427'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.984933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e1c15ae90e4b82b4e216b3a0549e2e', 'request_id': 'req-42277c9f-33cd-4d7d-8651-d0b9c1b63427'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:29.993926 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e1c15ae90e4b82b4e216b3a0549e2e', 'request_id': 'req-42277c9f-33cd-4d7d-8651-d0b9c1b63427'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.017567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] The token cache is being invalidate because project e5e1c15ae90e4b82b4e216b3a0549e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:30.019603 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.021065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-42277c9f-33cd-4d7d-8651-d0b9c1b63427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.029271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.029717 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1305/2614] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:29 2022] DELETE /identity/v3/projects/e5e1c15ae90e4b82b4e216b3a0549e2e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:30.030003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.031317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.058714 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:30.060040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:30.099566 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:30.135079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1310/2615] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:30.145981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.146267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.146545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.168222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:30.170214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:30.181222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1306/2616] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:30.185903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.198388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6983b5-cd79-40cc-84ee-bf275f8172ab None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.203515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.204232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.204636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.205001 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.205858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:30.206271 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.208093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.226706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1311/2617] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:37:30.230645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.244144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c25450-356c-4f23-abf4-53986b1ceb12 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.250992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.251706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.252214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.252647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.253520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.255841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.300641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1307/2618] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:37:30.305031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.327776 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c752-1766-4082-85a3-e9499e0ad474 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.421022 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.421743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.422105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.422442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.423160 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.425192 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.449458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1312/2619] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:30.453448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.465742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1b76-eebf-431c-91a0-867d28395ba7 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.472466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.473306 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.473687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.474063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.474821 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.476661 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.492413 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1308/2620] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:30.494397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.508654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df930e-63c2-4382-8415-c64467208852 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.516636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.517678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.518072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.518430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.519409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.521134 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ba3a9-00ae-4b7c-a7fd-c6964194e39a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.536505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1313/2621] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:30.538126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.551382 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c9e1-8148-45bd-ab1f-dc53bfa15cbf admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.556360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.557188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.557560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.557930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] PATH_INFO: `/v3/projects/2413e30aa42a4692a06ca89ba718407e/users/629f7f89439845d68027dbf4731756e8/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.586347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2413e30aa42a4692a06ca89ba718407e, user_id=629f7f89439845d68027dbf4731756e8, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.589563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.612010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1309/2622] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:30 2022] PUT /identity/v3/projects/2413e30aa42a4692a06ca89ba718407e/users/629f7f89439845d68027dbf4731756e8/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:30.624777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.625202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.628110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.679589 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] MFA Rules not processed for user `629f7f89439845d68027dbf4731756e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:30.680850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:30.720401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.722932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:30.831783 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b519d-3acc-4ea4-a295-6ae7753b1627 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.842101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.843073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.843356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.843633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] PATH_INFO: `/v3/users/27a4761813fc45dd88de2e264e8022d5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.844890 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a4761813fc45dd88de2e264e8022d5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.847029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.924774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1314/2623] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:30.929161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.944104 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f3ed5-29e8-45e6-be95-587fbc245417 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:30.947916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a4761813fc45dd88de2e264e8022d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.949446 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a4761813fc45dd88de2e264e8022d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.949654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:30.950265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:30.950547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:30.950819 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:30.951650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:30.954469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:30.963866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a4761813fc45dd88de2e264e8022d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.964878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a4761813fc45dd88de2e264e8022d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.967540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:30.978954 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7001e1f0-e100-40de-9813-d7f9353fd281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a4761813fc45dd88de2e264e8022d5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:30.993805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1315/2624] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:30.998108 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:30.999740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.010154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1310/2625] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:30 2022] DELETE /identity/v3/users/27a4761813fc45dd88de2e264e8022d5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:31.011775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.011966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.012296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.014267 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:31.043088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:31.044914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.088379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ee5fb-2d24-4d02-960c-2ae84198ef59 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.138495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1311/2626] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:31.147870 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.150766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.170836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.173689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.173689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.173689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.173689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.174811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.205520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1316/2627] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:30 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:31.207974 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.219142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcdc3-bced-4019-bcc8-1b0929517ebc admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.225708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.226650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.227035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.227900 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.228860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.230594 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.272575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1317/2628] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:31.274051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8ff3-e6dd-4b35-a6c2-6795fdd60bed None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.275791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.278322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.279312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.279750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.280178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.291846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b690f-e153-45a2-90fc-befff83ba3e9 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.297242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.298030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.299223 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.299392 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.299664 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] PATH_INFO: `/v3/projects/fe85327e45f344edbbaed133bb0ba198/users/c70df34913f04dc592aadfd8b9d6bea0/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.300176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.312411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1312/2629] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:31.323302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe85327e45f344edbbaed133bb0ba198, user_id=c70df34913f04dc592aadfd8b9d6bea0, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.326237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.352371 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1318/2630] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:31 2022] PUT /identity/v3/projects/fe85327e45f344edbbaed133bb0ba198/users/c70df34913f04dc592aadfd8b9d6bea0/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:31.358229 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.377880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b5a85-ee7b-4475-8cb7-e37b3efeea0c service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.464514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:31.478373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.479315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.479752 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.480406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.481339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.483867 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47cff-883a-453a-bdd7-f4159e93bb81 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.498311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1313/2631] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:31.501416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.510650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7005a4-f1cb-429d-90e3-954bd43c2391 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.517151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.518357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.518739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.519333 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] PATH_INFO: `/v3/projects/fe85327e45f344edbbaed133bb0ba198/users/c70df34913f04dc592aadfd8b9d6bea0/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.521469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe85327e45f344edbbaed133bb0ba198, user_id=c70df34913f04dc592aadfd8b9d6bea0, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.527304 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.552023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1319/2632] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:31 2022] PUT /identity/v3/projects/fe85327e45f344edbbaed133bb0ba198/users/c70df34913f04dc592aadfd8b9d6bea0/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:31.554547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.554547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.554844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.612277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] MFA Rules not processed for user `c70df34913f04dc592aadfd8b9d6bea0`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:31.613623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.677093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.679252 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.721715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1314/2633] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:31.732928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.734619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.772914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a124dc-30ae-4fcf-ad20-a9abca3f095c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.795560 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.797117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.798434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.798926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] PATH_INFO: `/v3/projects/5d9006a30c084e3cb22610b649cb5ed2` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.801068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9006a30c084e3cb22610b649cb5ed2)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.803167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.805973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:31.822156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffc78d-f218-4a83-917d-7feb81604898 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:31.827537 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:31.829084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:31.829530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:31.829985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:31.832004 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.872190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9006a30c084e3cb22610b649cb5ed2', 'request_id': 'req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:31.882341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9006a30c084e3cb22610b649cb5ed2', 'request_id': 'req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:31.892163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9006a30c084e3cb22610b649cb5ed2', 'request_id': 'req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:31.910305 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] The token cache is being invalidate because project 5d9006a30c084e3cb22610b649cb5ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:31.910692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:31.911522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:31.913433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1320/2634] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:31 2022] DELETE /identity/v3/projects/5d9006a30c084e3cb22610b649cb5ed2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:31.917456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:31.920156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:31.922575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:31.925085 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:31.939475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1315/2635] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:37:32.014985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ea59-0cf5-4712-aad5-1a7f5ed3279d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:32.014985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:32.014985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:32.014985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:32.015465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] PATH_INFO: `/v3/users/28fd7c9270664829ba2efab602e6f2b3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:32.015465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28fd7c9270664829ba2efab602e6f2b3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:32.015465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:32.079903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28fd7c9270664829ba2efab602e6f2b3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.091848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28fd7c9270664829ba2efab602e6f2b3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.092835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28fd7c9270664829ba2efab602e6f2b3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.112740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28fd7c9270664829ba2efab602e6f2b3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.113294 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28fd7c9270664829ba2efab602e6f2b3'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.144417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1321/2636] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:31 2022] DELETE /identity/v3/users/28fd7c9270664829ba2efab602e6f2b3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:32.693521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:32.711422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453ce69-4e77-4903-8392-05d5b512494c service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:32.801930 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:32.838862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:32.840048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:32.840048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:32.840234 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] PATH_INFO: `/v3/projects/36414b20dc9745bf80f6fa0692b3f656` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:32.842056 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36414b20dc9745bf80f6fa0692b3f656)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:32.844074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:32.910984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36414b20dc9745bf80f6fa0692b3f656', 'request_id': 'req-70cfed80-1c2e-4141-a10e-a48fe36f1132'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.920503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36414b20dc9745bf80f6fa0692b3f656', 'request_id': 'req-70cfed80-1c2e-4141-a10e-a48fe36f1132'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.927166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36414b20dc9745bf80f6fa0692b3f656', 'request_id': 'req-70cfed80-1c2e-4141-a10e-a48fe36f1132'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.944601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] The token cache is being invalidate because project 36414b20dc9745bf80f6fa0692b3f656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:32.944888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.945541 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:32.950098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1316/2637] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:32 2022] DELETE /identity/v3/projects/36414b20dc9745bf80f6fa0692b3f656 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:32.951282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:32.952865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:32.967897 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4f3-dbc9-4325-9cbd-72dc980d837f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:32.976522 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:33.046908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:33.047645 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:33.047995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:33.048368 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] PATH_INFO: `/v3/users/36842815057a4fee93d752494d881546` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:33.051492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36842815057a4fee93d752494d881546)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:33.057646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:33.131099 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36842815057a4fee93d752494d881546'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.141918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36842815057a4fee93d752494d881546'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.143359 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36842815057a4fee93d752494d881546'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.162710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36842815057a4fee93d752494d881546'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.163445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-f299245e-113d-4846-b6a1-492d3173b505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36842815057a4fee93d752494d881546'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.194163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1322/2638] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:37:32 2022] DELETE /identity/v3/users/36842815057a4fee93d752494d881546 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:33.572379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:33.573986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:33.596155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfed80-1c2e-4141-a10e-a48fe36f1132 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:33.679535 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:33.680210 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:33.680513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:33.681089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] PATH_INFO: `/v3/projects/51847fb43ee2475dabec6d0d968b1d67` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:33.681915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51847fb43ee2475dabec6d0d968b1d67)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:33.683839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:33.777321 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51847fb43ee2475dabec6d0d968b1d67', 'request_id': 'req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.819422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51847fb43ee2475dabec6d0d968b1d67', 'request_id': 'req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.831666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:33.852178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51847fb43ee2475dabec6d0d968b1d67', 'request_id': 'req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.872532 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] The token cache is being invalidate because project 51847fb43ee2475dabec6d0d968b1d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:37:33.872822 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.873812 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-e2345ddb-0712-4166-ae9a-fb2f30ebbac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:37:33.883275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1317/2639] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:37:33 2022] DELETE /identity/v3/projects/51847fb43ee2475dabec6d0d968b1d67 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:33.888032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:33.888206 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:33.888508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:33.930599 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:33.932299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.041973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1323/2640] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:34.050940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.051295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.051643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.073767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:34.075984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.088455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1318/2641] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:34.089505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.103399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac434b-fe82-4f4c-baf9-f319918b0389 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.107846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.108595 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.108937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.109307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.110021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:37:34.110385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.112468 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.126341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1324/2642] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:37:34.130411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.140726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ca228-a1ea-4e16-8b14-d2b780969d29 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.148189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.148884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.149157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.149424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.150073 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.152130 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.189810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1319/2643] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:37:34 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:34.191477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.202466 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c0a31-6bd5-4c9e-91f5-cbd85585057c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.219145 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:34.276911 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.277553 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.278064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.278469 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.279203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.281349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.308272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1325/2644] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:37:34 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:37:34.315918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.327750 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684737e4-308e-471b-8f5e-b3ee0a37de96 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.333681 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.334304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.334577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.334842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.335478 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.337097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.351391 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1320/2645] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:34.352880 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.367136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062e266-f7db-426d-9846-12aa7bc6ffca admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.372420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.373028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.373309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.373578 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.375699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.378011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fd540-c68f-49df-bb97-a873c69a070b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.390015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1326/2646] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:37:34.394170 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.405429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0626bc7-cbbc-43bd-9527-8564a72c2f88 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.410480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.411149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.411428 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.411703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] PATH_INFO: `/v3/projects/7c74432f09664669ba8008b92d8eb64b/users/791e0889e9c143c09ce09a231cc5f40e/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.428657 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c74432f09664669ba8008b92d8eb64b, user_id=791e0889e9c143c09ce09a231cc5f40e, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.431510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.448103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1321/2647] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:37:34 2022] PUT /identity/v3/projects/7c74432f09664669ba8008b92d8eb64b/users/791e0889e9c143c09ce09a231cc5f40e/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:37:34.459189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71caa9-bd77-4724-aa7c-277f4cbf5ad8 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.459726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71caa9-bd77-4724-aa7c-277f4cbf5ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.460151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71caa9-bd77-4724-aa7c-277f4cbf5ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.493093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fa71caa9-bd77-4724-aa7c-277f4cbf5ad8 None None] MFA Rules not processed for user `791e0889e9c143c09ce09a231cc5f40e`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:34.494450 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fa71caa9-bd77-4724-aa7c-277f4cbf5ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.572122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1327/2648] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:34.584270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.586158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.663482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865394d0-4165-4224-ab3b-2758c4366695 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.668204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.669138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.669488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.669833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.681842 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:34.683394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:34.696598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1322/2649] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:36.601790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020da68-e81e-49a6-aba8-41d5f99660ea None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:36.602288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020da68-e81e-49a6-aba8-41d5f99660ea None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:36.602387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020da68-e81e-49a6-aba8-41d5f99660ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:36.620921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-d020da68-e81e-49a6-aba8-41d5f99660ea None None] MFA Rules not processed for user `12f9f5e7d546480ea24913421273d9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:36.622232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d020da68-e81e-49a6-aba8-41d5f99660ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:36.676758 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1328/2650] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:37:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:37.063450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:37.065443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:37.160209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b65cb7-7bd5-4407-94d7-e61b1d336969 service glance] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:37.164083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:37.164728 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:37.165047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:37.165318 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:37.167121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:37.194913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:37.240525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:37.242022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:37.253063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1323/2651] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:41.984707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:41.985224 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:41.985888 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:42.011901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:42.013142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:42.047541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:42.079597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1329/2652] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:42.093087 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:42.095415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:42.176810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99474d93-f59c-45ea-9977-a09fa6a5cd57 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:42.182351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:42.183092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:42.183380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:42.183654 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:42.193650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:42.195342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ed78d-f799-4b3f-8de2-a8612c4c7c50 service swift] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:42.204216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1324/2653] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:37:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:37:52.072513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=11 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:52.200677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=50 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:06.038407 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:06.039839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.116680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b3b6-b0e9-4652-a0f7-b021d1a29927 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:06.121564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:06.122527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.122527 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.122836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] PATH_INFO: `/v3/users/629f7f89439845d68027dbf4731756e8` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.124005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629f7f89439845d68027dbf4731756e8)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:06.125935 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:06.175016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629f7f89439845d68027dbf4731756e8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.182531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629f7f89439845d68027dbf4731756e8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.183427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629f7f89439845d68027dbf4731756e8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.192055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629f7f89439845d68027dbf4731756e8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.192741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629f7f89439845d68027dbf4731756e8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.215424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1330/2654] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:06 2022] DELETE /identity/v3/users/629f7f89439845d68027dbf4731756e8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:06.220100 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.220674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.220674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.240483 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:06.241671 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.301180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1325/2655] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:06.311688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:06.313144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.372904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f7c-d3e6-4150-8f40-eed46389c799 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:06.377176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:06.378123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.378364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.378761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.388869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:06.390289 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:06.399705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1331/2656] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:06.661329 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:06.662979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.680682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687261-7a75-4b51-a8cb-7933f9e17453 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:06.686152 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:06.686792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.687075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.687353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] PATH_INFO: `/v3/projects/2413e30aa42a4692a06ca89ba718407e` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.688523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2413e30aa42a4692a06ca89ba718407e)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:06.690452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:06.738081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2413e30aa42a4692a06ca89ba718407e', 'request_id': 'req-bcb49556-e7ef-4d49-bea0-de7a87d55de2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.745264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2413e30aa42a4692a06ca89ba718407e', 'request_id': 'req-bcb49556-e7ef-4d49-bea0-de7a87d55de2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.751605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2413e30aa42a4692a06ca89ba718407e', 'request_id': 'req-bcb49556-e7ef-4d49-bea0-de7a87d55de2'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.762470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] The token cache is being invalidate because project 2413e30aa42a4692a06ca89ba718407e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:06.762795 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.764137 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.765647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1326/2657] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:06 2022] DELETE /identity/v3/projects/2413e30aa42a4692a06ca89ba718407e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:06.769736 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:06.779788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941d4d-c7d5-47bc-ae30-50b0f131dac2 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:06.798065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:06.847156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:06.847796 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.848075 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.848428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] PATH_INFO: `/v3/users/c70df34913f04dc592aadfd8b9d6bea0` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.849590 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c70df34913f04dc592aadfd8b9d6bea0)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:06.851519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:06.919390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c70df34913f04dc592aadfd8b9d6bea0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.929686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c70df34913f04dc592aadfd8b9d6bea0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.930394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c70df34913f04dc592aadfd8b9d6bea0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.940678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c70df34913f04dc592aadfd8b9d6bea0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.941751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a2e3e819-2628-4b4e-a658-fb568d04ac59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c70df34913f04dc592aadfd8b9d6bea0'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:06.971373 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1332/2658] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:06 2022] DELETE /identity/v3/users/c70df34913f04dc592aadfd8b9d6bea0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:07.348650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.350380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:07.384838 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49556-e7ef-4d49-bea0-de7a87d55de2 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.447740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.448398 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.448674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.448944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] PATH_INFO: `/v3/projects/fe85327e45f344edbbaed133bb0ba198` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.450076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe85327e45f344edbbaed133bb0ba198)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.452702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.483706 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:07.503158 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe85327e45f344edbbaed133bb0ba198', 'request_id': 'req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:07.511646 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe85327e45f344edbbaed133bb0ba198', 'request_id': 'req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:07.518498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe85327e45f344edbbaed133bb0ba198', 'request_id': 'req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:07.530092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] The token cache is being invalidate because project fe85327e45f344edbbaed133bb0ba198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:07.530527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:07.531244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-8ccc86e0-43d1-4c7f-bcdc-c591d80700fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:07.532845 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1327/2659] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:07 2022] DELETE /identity/v3/projects/fe85327e45f344edbbaed133bb0ba198 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:07.537715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.538210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.538311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.557442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:07.558639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:07.621955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1333/2660] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:07.630380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.630670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.630944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.650421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:07.651609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:07.660115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1328/2661] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:07.664283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.675013 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27895fe-3065-4a0a-8e29-c62399390761 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.679475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.680066 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.680402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.680679 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.681315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:07.681615 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.683194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.694976 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1334/2662] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:07.699163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.708554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25bd02-bf78-4ce5-a00d-872a05396c66 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.713201 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.713880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.714156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.714429 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.715063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.717680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.744477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1329/2663] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:07 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:07.750375 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.760361 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435d3a6-bd59-47bd-a27d-2143102031c4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.786949 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:07.818016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.818764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.819050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.819325 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.819952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.821882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.842962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1335/2664] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:07 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:07.847459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.857968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14c31-8d81-481a-929e-aa19287df112 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.862567 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.863151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.863432 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.863703 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.864380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.865996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.879006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1330/2665] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:07.882979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.893562 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3c3d-747b-49a3-b888-303fbc17a50f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.898205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.898797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.899078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.899351 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.899986 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.901605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb355f6-bdbd-4b39-9652-1e25deff3a3e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.912611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1336/2666] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:07.916637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:07.927204 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb208fe-5746-40e5-bb56-0929d0961aa5 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:07.931745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:07.932430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.932702 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.932971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] PATH_INFO: `/v3/projects/88f1caaea93245c1be31aba6ad85a113/users/9ace3ba23d874369b71c6619f37dfdb4/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:07.947699 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f1caaea93245c1be31aba6ad85a113, user_id=9ace3ba23d874369b71c6619f37dfdb4, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.950909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.964490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1331/2667] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:07 2022] PUT /identity/v3/projects/88f1caaea93245c1be31aba6ad85a113/users/9ace3ba23d874369b71c6619f37dfdb4/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:07.975413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:07.975702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:07.976003 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.003148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] MFA Rules not processed for user `9ace3ba23d874369b71c6619f37dfdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:08.004369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:08.066515 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1337/2668] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:08.071043 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.133420 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e7cc-9435-4d9d-8e6f-282397397e00 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.138570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.139168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.139445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.139718 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.140448 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.142334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.172921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1332/2669] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:08 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:08.174014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.186230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862d5a-d4e1-4a2d-9a39-7bb1d4ae6d1b None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.244760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.245455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.245799 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.246136 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.246835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.248917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.272427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1338/2670] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:08 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:08.275066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.286499 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac374e-b685-4550-9d67-4a747118539d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.300711 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.301332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.301608 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.301877 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.302611 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.304238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.316150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1333/2671] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:08.321153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.332383 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775050-f87f-40f7-ab1a-181158aed7f2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.337370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.338032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.338311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.338581 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] PATH_INFO: `/v3/projects/0f6f79ce3dc14aaea2001c3dae395376/users/af1c995c5579434ca70f542099567dd1/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.358699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6f79ce3dc14aaea2001c3dae395376, user_id=af1c995c5579434ca70f542099567dd1, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.361429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.373969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1339/2672] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:08 2022] PUT /identity/v3/projects/0f6f79ce3dc14aaea2001c3dae395376/users/af1c995c5579434ca70f542099567dd1/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:08.377957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.388292 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded459f-32bd-4f2a-b268-14dfd2482819 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.408503 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:08.446689 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.447301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.447582 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.447859 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.448530 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.450123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7aba2e-d84b-4ebe-aa88-69ff4c5dc7d1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.460191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1334/2673] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:08.464572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.470577 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:08.477835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e336998-3f6c-4921-a2e0-ed2591bd2eed admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.485943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.486706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.487043 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.487381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] PATH_INFO: `/v3/projects/0f6f79ce3dc14aaea2001c3dae395376/users/af1c995c5579434ca70f542099567dd1/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.489390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6f79ce3dc14aaea2001c3dae395376, user_id=af1c995c5579434ca70f542099567dd1, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.492016 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.502307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1340/2674] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:08 2022] PUT /identity/v3/projects/0f6f79ce3dc14aaea2001c3dae395376/users/af1c995c5579434ca70f542099567dd1/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:08.512581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.513476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.513810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.542023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] MFA Rules not processed for user `af1c995c5579434ca70f542099567dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:08.543212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:08.608575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1335/2675] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:08.620730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:08.622470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:08.678301 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e28a2-c378-4c24-9b24-b49ae7cc63fc admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:08.681948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:08.682538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:08.682812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:08.683078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:08.691920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:08.693288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:08.702985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1341/2676] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:09.010080 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.019807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c7f9-9b78-418a-a307-0b533b413d3d None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.024307 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af1c995c5579434ca70f542099567dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f6f79ce3dc14aaea2001c3dae395376', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.024894 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.025176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.025452 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.026084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:09.026381 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.028030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.039297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1336/2677] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:09.043153 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.052456 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c4734-42e9-4bb6-8efb-d06a2e09beae service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.057070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af1c995c5579434ca70f542099567dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f6f79ce3dc14aaea2001c3dae395376', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.057660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.057939 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.058210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.058836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.061390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.087957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1342/2678] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:09 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:09.092670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.102738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbe3b-433b-4bc3-87b0-1c374c2be0a5 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.158490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af1c995c5579434ca70f542099567dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f6f79ce3dc14aaea2001c3dae395376', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.159082 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.159358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.159630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.160330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:09.160633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.162216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.175279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1337/2679] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:09.211232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.222501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4878ba-1406-4164-9c30-ba5a5c501058 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.226966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af1c995c5579434ca70f542099567dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f6f79ce3dc14aaea2001c3dae395376', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.227583 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.227865 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.228137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] PATH_INFO: `/v3/projects/a50e94f66cf4497fb28cd329e823faf2` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.229310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a50e94f66cf4497fb28cd329e823faf2)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.231969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.277467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50e94f66cf4497fb28cd329e823faf2', 'request_id': 'req-15c2b8e9-7b75-4456-a9b1-805af7a2878b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.285109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50e94f66cf4497fb28cd329e823faf2', 'request_id': 'req-15c2b8e9-7b75-4456-a9b1-805af7a2878b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.293040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50e94f66cf4497fb28cd329e823faf2', 'request_id': 'req-15c2b8e9-7b75-4456-a9b1-805af7a2878b'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.303634 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] The token cache is being invalidate because project a50e94f66cf4497fb28cd329e823faf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:09.303927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.304582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-15c2b8e9-7b75-4456-a9b1-805af7a2878b tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.306088 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1343/2680] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:09 2022] DELETE /identity/v3/projects/a50e94f66cf4497fb28cd329e823faf2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:09.523011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.524834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:09.556470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:09.591933 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a7316-bc57-4f25-98c2-f6a1bc001440 tempest-VolumeQuotaClassesTest-103995014 tempest-VolumeQuotaClassesTest-103995014-project-admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.647215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.648015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.648315 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.648594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] PATH_INFO: `/v3/users/9ace3ba23d874369b71c6619f37dfdb4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.650058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ace3ba23d874369b71c6619f37dfdb4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.651957 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.702356 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ace3ba23d874369b71c6619f37dfdb4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.703063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ace3ba23d874369b71c6619f37dfdb4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.710364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ace3ba23d874369b71c6619f37dfdb4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.711028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ace3ba23d874369b71c6619f37dfdb4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.718310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ace3ba23d874369b71c6619f37dfdb4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:09.735581 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1338/2681] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:09 2022] DELETE /identity/v3/users/9ace3ba23d874369b71c6619f37dfdb4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:09.740056 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.740544 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.740641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.759518 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:09.760673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:09.806042 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:09.820521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1344/2682] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:09.831029 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.832480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:09.892123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311a5eb-dae7-4e82-8afb-c2ec5b81ee1a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.895707 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.896323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.896600 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.897034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.904415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:09.906860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:09.908590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:09.920623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1339/2683] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:10.188426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:10.189929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:10.201938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57fc39-9fa7-4ffd-889b-bf48a6f9aadc None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:10.206614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:10.207197 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:10.207483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:10.207751 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] PATH_INFO: `/v3/projects/88f1caaea93245c1be31aba6ad85a113` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:10.208896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f1caaea93245c1be31aba6ad85a113)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:10.210779 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:10.257441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f1caaea93245c1be31aba6ad85a113', 'request_id': 'req-99ac2932-aeaf-4b6c-ab72-f18030e991db'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.263824 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f1caaea93245c1be31aba6ad85a113', 'request_id': 'req-99ac2932-aeaf-4b6c-ab72-f18030e991db'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.272409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f1caaea93245c1be31aba6ad85a113', 'request_id': 'req-99ac2932-aeaf-4b6c-ab72-f18030e991db'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.283602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] The token cache is being invalidate because project 88f1caaea93245c1be31aba6ad85a113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:10.283892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.284576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.286245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1345/2684] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:10 2022] DELETE /identity/v3/projects/88f1caaea93245c1be31aba6ad85a113 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:10.290247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:10.300414 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397d38-0860-40e5-ac30-868fd8bf1634 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:10.357166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:10.357810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:10.358090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:10.358365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] PATH_INFO: `/v3/users/af1c995c5579434ca70f542099567dd1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:10.359494 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1c995c5579434ca70f542099567dd1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:10.361501 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:10.407416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1c995c5579434ca70f542099567dd1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.408066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1c995c5579434ca70f542099567dd1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.414880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1c995c5579434ca70f542099567dd1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.415573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1c995c5579434ca70f542099567dd1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.423403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-89144232-6d31-4fd7-accc-433a978d05c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1c995c5579434ca70f542099567dd1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.441379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1340/2685] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:10 2022] DELETE /identity/v3/users/af1c995c5579434ca70f542099567dd1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:10.761426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:10.762945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:10.777320 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2932-aeaf-4b6c-ab72-f18030e991db admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:10.844349 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:10.845006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:10.845288 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:10.845568 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] PATH_INFO: `/v3/projects/0f6f79ce3dc14aaea2001c3dae395376` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:10.847464 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6f79ce3dc14aaea2001c3dae395376)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:10.849917 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:10.905261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6f79ce3dc14aaea2001c3dae395376', 'request_id': 'req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.912354 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6f79ce3dc14aaea2001c3dae395376', 'request_id': 'req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.921131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6f79ce3dc14aaea2001c3dae395376', 'request_id': 'req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.932715 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] The token cache is being invalidate because project 0f6f79ce3dc14aaea2001c3dae395376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:10.933093 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.933814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c0b2ab4b-e47e-406e-bd1f-2f88f7b89ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:10.935769 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1346/2686] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:10 2022] DELETE /identity/v3/projects/0f6f79ce3dc14aaea2001c3dae395376 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:10.940806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:10.941277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:10.941486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:10.962735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:10.964048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.030263 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1341/2687] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:11.039841 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.040127 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.040438 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.056467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:11.064437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:11.066505 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.074512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1347/2688] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:11.079072 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.089736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10f11b-e353-47dd-852f-b1837b95417b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.096006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.096641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.096916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.097188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.097831 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:11.098131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.099830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.115515 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1342/2689] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:11.116652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.126410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d902d-9ec8-41b4-8556-24dbaf4d082f None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.131270 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.131862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.132139 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.132445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.133072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.134970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.161761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1348/2690] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:11 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:11.165928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.171856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:11.177929 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610bdaa-c60c-4b65-8f2a-c4a757785a2d admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.234455 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.235058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.235339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.235615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.236270 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.238188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.258520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1343/2691] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:11 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:11.262693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.272822 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465b87-d0d9-4332-b88b-aa17bc83c036 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.277313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.277904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.278688 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.278942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.279575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.281199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.291628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1349/2692] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:11.295663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.305459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f917106-85eb-49d5-8ef0-43fbe605910f admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.310759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.311285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.311560 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.311830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.312497 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.314070 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b484b1-794d-4604-9c93-511e47ee171f admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.324332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1344/2693] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:11.328847 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.339105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdae364-d7cc-438e-9301-252a3225b74c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.343772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.344443 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.344719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.344987 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] PATH_INFO: `/v3/projects/90be8c19987f494ab07cc2d634025635/users/c3aa6ce8af514240bd3e084b7e3d4c78/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.359446 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90be8c19987f494ab07cc2d634025635, user_id=c3aa6ce8af514240bd3e084b7e3d4c78, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.362133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.375741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1350/2694] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:11 2022] PUT /identity/v3/projects/90be8c19987f494ab07cc2d634025635/users/c3aa6ce8af514240bd3e084b7e3d4c78/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:11.386353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.386641 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.386915 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.415167 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] MFA Rules not processed for user `c3aa6ce8af514240bd3e084b7e3d4c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:11.416390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.480898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1345/2695] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:11.493025 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.494719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.555242 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de23243-6f26-47a0-bb25-20952ded5400 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.559313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.559952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.560226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.560534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.569184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.570600 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717cc47-a8ef-47e2-8f41-96b2803bf27b service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.580537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1351/2696] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:38:11.603025 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.687377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee7f7-e8ec-4279-91cd-b62c61c49f99 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.698110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.698864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.701774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.702174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] PATH_INFO: `/v3/users/c3aa6ce8af514240bd3e084b7e3d4c78` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.710741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3aa6ce8af514240bd3e084b7e3d4c78)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.710741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:11.723589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:11.765311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3aa6ce8af514240bd3e084b7e3d4c78'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:11.766211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3aa6ce8af514240bd3e084b7e3d4c78'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:11.779899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3aa6ce8af514240bd3e084b7e3d4c78'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:11.780792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3aa6ce8af514240bd3e084b7e3d4c78'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:11.789045 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3aa6ce8af514240bd3e084b7e3d4c78'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:11.813242 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1346/2697] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:11 2022] DELETE /identity/v3/users/c3aa6ce8af514240bd3e084b7e3d4c78 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:11.817642 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.818039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.818179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.843585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:11.844881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.907827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1352/2698] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:11.917624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:11.919278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:11.977993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc23f-56f4-453f-b820-494384693de3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:11.981993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b326c-da01-4709-b662-471534046310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:11.982594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b326c-da01-4709-b662-471534046310 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:11.982868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b326c-da01-4709-b662-471534046310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:11.983135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b326c-da01-4709-b662-471534046310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:11.992570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b326c-da01-4709-b662-471534046310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:11.993996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b326c-da01-4709-b662-471534046310 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.004566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1347/2699] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:12.283210 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.285198 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.308730 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accd852-51a5-4692-b381-03cde3cf1787 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.314426 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.323250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.323250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.323250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] PATH_INFO: `/v3/projects/90be8c19987f494ab07cc2d634025635` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.323250 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90be8c19987f494ab07cc2d634025635)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.323651 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.363952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:12.371474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90be8c19987f494ab07cc2d634025635', 'request_id': 'req-4bf2547f-8763-405c-b945-9b8c3ddba99e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:12.379377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90be8c19987f494ab07cc2d634025635', 'request_id': 'req-4bf2547f-8763-405c-b945-9b8c3ddba99e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:12.387026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90be8c19987f494ab07cc2d634025635', 'request_id': 'req-4bf2547f-8763-405c-b945-9b8c3ddba99e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:12.397459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] The token cache is being invalidate because project 90be8c19987f494ab07cc2d634025635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:12.397747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:12.398360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-4bf2547f-8763-405c-b945-9b8c3ddba99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:12.399829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1353/2700] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:12 2022] DELETE /identity/v3/projects/90be8c19987f494ab07cc2d634025635 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:12.404748 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.405033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.405308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.424823 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:12.426011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.485293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1348/2701] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:12.493337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.493622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.493896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.513644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:12.514942 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.522961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1354/2702] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:12.526948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.536639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef71c-3a1d-4700-b671-37d0dcaf57b2 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.541123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.541712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.541984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.542252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.542880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:12.543175 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.545288 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.556757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1349/2703] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:12.561756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.570946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce028d-83bb-48dc-bf27-6c06d48f3270 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.575696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.576287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.576569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.576834 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.577455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.579298 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.606901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1355/2704] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:12.611135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.621225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e1d1-c26a-4514-b71c-52c058db6144 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.650377 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:12.696730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.697330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.697605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.697871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.698528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.700589 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.728506 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1350/2705] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:12.734048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.744433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126dfd-6b4b-4aac-961c-98b682c38768 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.749233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.749843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.750118 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.750388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.751014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.752657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.763749 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1356/2706] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:12.767519 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.780623 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c417865-b1d1-4a0c-9766-b13d99b9e183 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.787369 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.788476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.788757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.789035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.790071 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.795422 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677fbf5-3efa-4e90-b321-f633f98e1c51 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.820335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1351/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:12.822988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.833591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0415-e559-401b-b6ce-b5341925d58a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.838237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:12.838885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.839348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.839572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] PATH_INFO: `/v3/projects/0bbc1b0230a949c19ff51a3e9335ac8d/users/bd7e86dc187e47cfb524e4cf8374bc6a/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.861278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bbc1b0230a949c19ff51a3e9335ac8d, user_id=bd7e86dc187e47cfb524e4cf8374bc6a, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:12.863969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:12.881478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1357/2708] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:12 2022] PUT /identity/v3/projects/0bbc1b0230a949c19ff51a3e9335ac8d/users/bd7e86dc187e47cfb524e4cf8374bc6a/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:12.893480 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.893765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.894035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.935684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] MFA Rules not processed for user `bd7e86dc187e47cfb524e4cf8374bc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:12.937034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.965428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.968475 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:13.038342 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076525-4e03-45bd-b2b5-db2337262790 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.038592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1352/2709] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:13.042286 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.042476 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.043132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.043547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.044109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.048017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:13.119181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d7833-bb71-4a79-bdea-aa158278c12b None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.124183 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.125051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.126067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.126184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.126943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.128755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.129100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.130512 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.143324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1358/2710] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:13.160747 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1353/2711] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:13 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:13.161852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.172123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3960f5-7ca9-4fc0-81cf-dbce5aa9bc11 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.183310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:13.233343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.233947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.234225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.234491 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.235129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.237049 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.261810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1359/2712] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:13 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:13.263047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.273132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fdf6c-5c87-4b32-b7c1-73b6b3c55689 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.277791 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.278637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.278913 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.279180 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.279820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.281443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.295847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1354/2713] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:13.297133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.306985 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd071-3665-4f4d-a3ec-76648aba759a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.312313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.313021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.313310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.313564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.314184 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.315748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba086-0baa-4961-877c-f6f4af68363b admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.328296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1360/2714] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:13.331009 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.336858 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:13.344618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380d57-6dd5-4563-97c0-135edb8a9418 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.351028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.351769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.352047 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.352352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] PATH_INFO: `/v3/projects/7aabb8efe3014f3bba3d3ab66f65ec95/users/c9fe561e035a4070ac0da1cf2ae783a3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.369758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aabb8efe3014f3bba3d3ab66f65ec95, user_id=c9fe561e035a4070ac0da1cf2ae783a3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.372660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.387447 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1355/2715] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:13 2022] PUT /identity/v3/projects/7aabb8efe3014f3bba3d3ab66f65ec95/users/c9fe561e035a4070ac0da1cf2ae783a3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:13.397770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.398117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.398480 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.435278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] MFA Rules not processed for user `c9fe561e035a4070ac0da1cf2ae783a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:13.436161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:13.510073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1361/2716] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:13.515807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.583688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c926b2-d36a-4783-b3f8-746ba085bd4a admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.588467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.589074 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.589348 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.589617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.591495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.592179 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.624970 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.626570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1356/2717] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:13 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:13.636459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373525f-17fc-4044-b657-f6c3a4e7ca7a None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.698092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.698786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.699129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.699462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.700638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.702614 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.728914 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1362/2718] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:13 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:13.734910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.745326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb48f1-136b-4567-bef4-adaae02e07f9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.750475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.751153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.751458 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.751729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.752971 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.753993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.765450 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1357/2719] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:13.769435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.780110 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e9dd5-c381-4f51-84a0-f31af743fdc5 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.784782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.785433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.785712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.785984 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] PATH_INFO: `/v3/projects/22965960fb084c0190a880dceeec13d3/users/1f45c840fcd1428cb5621aa008b51efa/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.797753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:13.801964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=22965960fb084c0190a880dceeec13d3, user_id=1f45c840fcd1428cb5621aa008b51efa, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.805652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.820827 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1363/2720] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:13 2022] PUT /identity/v3/projects/22965960fb084c0190a880dceeec13d3/users/1f45c840fcd1428cb5621aa008b51efa/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:13.825792 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.836898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94991d-09f6-4ad8-a1b1-82536e2c5370 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.927638 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.928457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.928799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.929540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.930335 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.932568 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c5049-00e5-4495-a7ef-7c56b1425363 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.948285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1358/2721] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:13.952044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:13.968956 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dffd9-380c-4ad0-a507-c60c92c44aed admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:13.974998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.975797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.976176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.976658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] PATH_INFO: `/v3/projects/22965960fb084c0190a880dceeec13d3/users/1f45c840fcd1428cb5621aa008b51efa/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.979171 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22965960fb084c0190a880dceeec13d3, user_id=1f45c840fcd1428cb5621aa008b51efa, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.982728 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.995356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1364/2722] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:13 2022] PUT /identity/v3/projects/22965960fb084c0190a880dceeec13d3/users/1f45c840fcd1428cb5621aa008b51efa/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:14.008164 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:14.008507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:14.008788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:14.047434 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] MFA Rules not processed for user `1f45c840fcd1428cb5621aa008b51efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:14.049036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:14.102174 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:14.144393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1359/2723] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:14.156585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:14.159292 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:14.233878 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630df5e-f555-406b-9594-2ece07cd14b2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:14.240735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:14.241447 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:14.241804 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:14.242176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:14.329111 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:14.330943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:14.349941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1365/2724] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:15.971352 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:15.982618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf1939-6a54-43b3-ba9d-6d052dffe0a7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:15.986471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:15.987687 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:15.988311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:15.988533 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.052820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:16.054521 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700b30-5aa7-4903-8f4a-1de5ba02eb4b service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:16.065996 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1360/2725] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:16.445948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:16.447459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.568321 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efdd96-943d-4f80-b295-0e0589cc031a service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:16.573855 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:16.574636 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:16.574933 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:16.575209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] PATH_INFO: `/v3/users/791e0889e9c143c09ce09a231cc5f40e` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.576541 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791e0889e9c143c09ce09a231cc5f40e)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:16.578650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:16.631092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:16.654661 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791e0889e9c143c09ce09a231cc5f40e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.666775 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791e0889e9c143c09ce09a231cc5f40e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.667692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791e0889e9c143c09ce09a231cc5f40e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.676647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791e0889e9c143c09ce09a231cc5f40e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.677451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791e0889e9c143c09ce09a231cc5f40e'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.718324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1366/2726] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:16 2022] DELETE /identity/v3/users/791e0889e9c143c09ce09a231cc5f40e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:16.719837 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:16.720121 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:16.720423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.753470 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:16.755435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.833026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1361/2727] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:16.844674 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:16.846999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.880664 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:16.883847 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.902539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c227820-a0c5-4255-a7bb-eca0d0ccbf77 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:16.908133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:16.908793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:16.909089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:16.909336 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] PATH_INFO: `/v3/users/30ccb163613a40ac8042f229199717f6` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.910453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ccb163613a40ac8042f229199717f6)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:16.912386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:16.931981 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c5103-1fa4-4918-8d98-4faab9763814 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:16.935755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:16.936399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:16.936777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:16.937090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.947988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:16.949448 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:16.962713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ccb163613a40ac8042f229199717f6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.963403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ccb163613a40ac8042f229199717f6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.964236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1367/2728] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:16.969217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:16.971185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.973978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ccb163613a40ac8042f229199717f6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.975061 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ccb163613a40ac8042f229199717f6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.985176 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ccb163613a40ac8042f229199717f6'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:16.993372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:17.011765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1362/2729] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:16 2022] DELETE /identity/v3/users/30ccb163613a40ac8042f229199717f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:17.056915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d912a55-55e5-4770-af1d-5fcc20635134 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.119273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.119879 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.120604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.120604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.122825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:17.151390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:17.201626 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.204137 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.220837 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1368/2730] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:17.233394 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.235040 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:17.298401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.299877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:17.329424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a6ccd-366a-4ae3-8920-61395f4f3a1e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.338960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.339610 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.339889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.340161 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] PATH_INFO: `/v3/projects/49fe69e57e744eea93988aecfa9decc2` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.341771 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fe69e57e744eea93988aecfa9decc2)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.345097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.381081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62670fc-97fc-4da0-84db-8080818c8b8b service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.388202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.388937 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.389215 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.389520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] PATH_INFO: `/v3/projects/7c74432f09664669ba8008b92d8eb64b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.390923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c74432f09664669ba8008b92d8eb64b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.393070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.412679 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fe69e57e744eea93988aecfa9decc2', 'request_id': 'req-84cad6ed-f919-47c7-bced-8f4b0f899677'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.423444 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fe69e57e744eea93988aecfa9decc2', 'request_id': 'req-84cad6ed-f919-47c7-bced-8f4b0f899677'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.431341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fe69e57e744eea93988aecfa9decc2', 'request_id': 'req-84cad6ed-f919-47c7-bced-8f4b0f899677'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.443226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] The token cache is being invalidate because project 49fe69e57e744eea93988aecfa9decc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:17.443514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.444143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-84cad6ed-f919-47c7-bced-8f4b0f899677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.445736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1363/2731] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:17 2022] DELETE /identity/v3/projects/49fe69e57e744eea93988aecfa9decc2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:17.449502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c74432f09664669ba8008b92d8eb64b', 'request_id': 'req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.450805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.451177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.451459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.456510 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c74432f09664669ba8008b92d8eb64b', 'request_id': 'req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.464356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c74432f09664669ba8008b92d8eb64b', 'request_id': 'req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.474851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:17.475998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:17.478168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] The token cache is being invalidate because project 7c74432f09664669ba8008b92d8eb64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:17.478455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.479081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c93955dd-9f8e-4e6a-acc2-40923f6fc8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:17.480789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1369/2732] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:17 2022] DELETE /identity/v3/projects/7c74432f09664669ba8008b92d8eb64b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:17.545740 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1364/2733] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:17.553965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.554243 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.554513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.577992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:17.579204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:17.587239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1370/2734] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:17.591928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.602672 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de44f1-3efd-4636-8b8e-8b3eee0ad4d8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.608272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.608928 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.609200 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.609465 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.610115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:17.610411 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.612026 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.629430 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1365/2735] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:17.630554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.640109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead6aca-ad7c-410c-85d3-b6809fec19e6 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.645071 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.645707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.645988 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.646281 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.646930 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.648891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.678929 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1371/2736] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:17 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:17.683205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.689208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:17.695311 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2e3e7-71e9-46ac-af35-6dea4121f8b6 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.759209 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.759729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.760005 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.760322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.760959 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.763069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.785028 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1366/2737] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:17 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:17.789133 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.800044 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cd489-79f2-4a0f-9e82-7a3882ea2495 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.804710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.805293 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.805565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.805829 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.806458 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.808021 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.818580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1372/2738] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:17.822367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.832466 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f411b-449a-4c94-bcbe-4f2e4e332079 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.837084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.837668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.837945 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.838215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.838844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.841010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40c6f2-e656-4d21-b04f-5bcc422589d6 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.851609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1367/2739] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:17.855644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:17.865936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fa19e-dc88-43a8-b765-03ff804865c0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:17.873430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:17.874253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.874529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.875457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] PATH_INFO: `/v3/projects/5be2f7492aa340d29c57c6844e484d34/users/cc8607b63d1248d2a45a5d2d665f7eee/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.879870 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:17.896768 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be2f7492aa340d29c57c6844e484d34, user_id=cc8607b63d1248d2a45a5d2d665f7eee, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:17.899397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:17.912671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1373/2740] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:17 2022] PUT /identity/v3/projects/5be2f7492aa340d29c57c6844e484d34/users/cc8607b63d1248d2a45a5d2d665f7eee/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:17.924833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:17.925332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:17.925818 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:17.954076 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] MFA Rules not processed for user `cc8607b63d1248d2a45a5d2d665f7eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:17.955399 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:18.021944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1368/2741] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:18.026501 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.102772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c3a09-86c5-4ef1-a8bf-6da7c23c6928 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.112154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.112795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.113070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.113338 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.113979 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.115861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.143946 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1374/2742] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:18 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:18.148538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.160583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e263d-d919-44bc-b8ea-dd314cc310ee None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.217492 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.218092 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.218364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.218636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.219254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.221459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.237973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:18.243615 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1369/2743] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:18 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:18.247457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.257452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c93f-9f37-4213-8e74-e4358376f6b1 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.261977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.263083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.263347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.263610 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.264235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.265821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.276040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1375/2744] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:18.280736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.291150 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42149e4-e1da-42da-aedf-e137bab00677 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.296069 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.296777 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.297058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.297339 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] PATH_INFO: `/v3/projects/4182a390f6d84e18aa4f148daa094e92/users/77b39e5a227c4a43978e441af98bc069/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.313373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4182a390f6d84e18aa4f148daa094e92, user_id=77b39e5a227c4a43978e441af98bc069, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.316046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.336871 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1370/2745] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:18 2022] PUT /identity/v3/projects/4182a390f6d84e18aa4f148daa094e92/users/77b39e5a227c4a43978e441af98bc069/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:18.341409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.351698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c03b-37da-4026-ab6d-6e782689be7c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.419330 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.419945 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.420217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.420514 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.421141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.422716 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf4540-0176-464e-9acd-0ef3af2a8ecf admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.437073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1376/2746] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:18.438312 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.450115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19400d5-2083-4b62-adbe-a9082c446bc3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.454815 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.455459 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.455754 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.456020 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] PATH_INFO: `/v3/projects/4182a390f6d84e18aa4f148daa094e92/users/77b39e5a227c4a43978e441af98bc069/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.457950 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4182a390f6d84e18aa4f148daa094e92, user_id=77b39e5a227c4a43978e441af98bc069, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.460593 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.471882 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1371/2747] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:18 2022] PUT /identity/v3/projects/4182a390f6d84e18aa4f148daa094e92/users/77b39e5a227c4a43978e441af98bc069/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:18.483774 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.484311 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.484495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.515176 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] MFA Rules not processed for user `77b39e5a227c4a43978e441af98bc069`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:18.516478 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:18.562787 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:18.592036 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1377/2748] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:18.603873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.606889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:18.683729 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269378-4c28-44f5-b17c-83cb5b4e79ee admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.689238 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.689851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.690124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.690392 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.700575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.702030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a55e-32bd-49e3-88a4-df2de3c1b63e service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.711936 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1372/2749] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:18.851204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.915802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52eca-25c9-4db9-8a83-a816e69fec9c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.921101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:18.921726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.922004 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.922279 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] PATH_INFO: `/v3/users/cc8607b63d1248d2a45a5d2d665f7eee` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.923390 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8607b63d1248d2a45a5d2d665f7eee)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:18.925404 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:18.972297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8607b63d1248d2a45a5d2d665f7eee'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:18.979604 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8607b63d1248d2a45a5d2d665f7eee'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:18.980278 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8607b63d1248d2a45a5d2d665f7eee'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:18.987995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8607b63d1248d2a45a5d2d665f7eee'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:18.988690 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8607b63d1248d2a45a5d2d665f7eee'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.006436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1378/2750] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:18 2022] DELETE /identity/v3/users/cc8607b63d1248d2a45a5d2d665f7eee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:19.011022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:19.011297 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:19.011591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:19.039255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:19.040486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:19.098742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1373/2751] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:19.109112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:19.110576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:19.169155 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc53a7-9e45-495e-b1e0-d362262d5481 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:19.172755 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:19.173350 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:19.173622 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:19.173887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:19.182534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:19.184576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:19.193904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1379/2752] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:19.449899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:19.451372 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:19.464626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafe2b8-080f-4725-9c20-efd1b867c420 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:19.470096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:19.471058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:19.471338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:19.471614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] PATH_INFO: `/v3/projects/5be2f7492aa340d29c57c6844e484d34` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:19.472902 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be2f7492aa340d29c57c6844e484d34)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:19.475426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:19.523228 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be2f7492aa340d29c57c6844e484d34', 'request_id': 'req-0b2b9a14-adcb-419f-8a51-322274c3f521'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.531338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be2f7492aa340d29c57c6844e484d34', 'request_id': 'req-0b2b9a14-adcb-419f-8a51-322274c3f521'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.535188 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:19.537883 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be2f7492aa340d29c57c6844e484d34', 'request_id': 'req-0b2b9a14-adcb-419f-8a51-322274c3f521'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.550573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] The token cache is being invalidate because project 5be2f7492aa340d29c57c6844e484d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:19.550851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.551481 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.553015 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1374/2753] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:19 2022] DELETE /identity/v3/projects/5be2f7492aa340d29c57c6844e484d34 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:19.557081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:19.562542 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:19.568920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5594-74ff-4b49-bfd5-9bc4882ddbaf service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:19.625263 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:19.625881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:19.626154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:19.626420 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] PATH_INFO: `/v3/users/77b39e5a227c4a43978e441af98bc069` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:19.627517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b39e5a227c4a43978e441af98bc069)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:19.629485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:19.673809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b39e5a227c4a43978e441af98bc069'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.681347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b39e5a227c4a43978e441af98bc069'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.681950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b39e5a227c4a43978e441af98bc069'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.689838 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b39e5a227c4a43978e441af98bc069'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.690457 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3273097f-bc6a-4423-b6ed-6d4731d2e722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b39e5a227c4a43978e441af98bc069'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:19.708417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1380/2754] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:19 2022] DELETE /identity/v3/users/77b39e5a227c4a43978e441af98bc069 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:19.994982 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:19.997236 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:20.013558 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b9a14-adcb-419f-8a51-322274c3f521 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.087279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.088142 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.088310 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.088594 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] PATH_INFO: `/v3/projects/4182a390f6d84e18aa4f148daa094e92` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.089981 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4182a390f6d84e18aa4f148daa094e92)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.092998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.143135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4182a390f6d84e18aa4f148daa094e92', 'request_id': 'req-6c11476b-b617-463a-ae0c-ba85ec834799'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:20.150214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4182a390f6d84e18aa4f148daa094e92', 'request_id': 'req-6c11476b-b617-463a-ae0c-ba85ec834799'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:20.156327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4182a390f6d84e18aa4f148daa094e92', 'request_id': 'req-6c11476b-b617-463a-ae0c-ba85ec834799'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:20.167561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] The token cache is being invalidate because project 4182a390f6d84e18aa4f148daa094e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:20.167828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:20.169046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6c11476b-b617-463a-ae0c-ba85ec834799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:20.170663 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1375/2755] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:19 2022] DELETE /identity/v3/projects/4182a390f6d84e18aa4f148daa094e92 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:20.175394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.175680 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.175951 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.195672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:20.196896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:20.258142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1381/2756] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:20.266258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.266538 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.266807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.288879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:20.290094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:20.299003 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1376/2757] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:20.303245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.313788 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6885746-6316-4089-929b-78ed3e17cea5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.318328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.318922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.319199 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.319513 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.320130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:20.320459 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.322038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.333520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1382/2758] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:38:20.337816 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.347836 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923425e-83ef-40a4-b8e6-67ed451655a5 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.352477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.353064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.353334 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.353601 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.354226 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.357038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.390507 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1377/2759] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:20.394753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.405212 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e07c4-e622-4a39-a95a-fd418f8ba84e admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.410061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:20.464771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.466061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.466408 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.466706 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.467328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.469261 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.491329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1383/2760] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:20.495265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.505757 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319214df-e683-4b43-a5bb-1db936adeadc admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.510742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.511360 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.511636 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.512454 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.513093 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.514690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.524606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1378/2761] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:20.529122 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.538959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c0ef-dd68-440b-8be9-92d134f5a2fe admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.543559 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.544141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.544455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.544739 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.545372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.546967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ebd87-b02f-4f36-a1f0-75b095ac867a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.557360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1384/2762] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:20.561848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.571604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4392c-947b-466b-be5c-214a15f830fe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.576684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.577325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.577602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.577878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] PATH_INFO: `/v3/projects/7014ddb5f34d4915bb3def14c2814e5f/users/b65ce68ef01646a1a324419da49276cf/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.592598 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7014ddb5f34d4915bb3def14c2814e5f, user_id=b65ce68ef01646a1a324419da49276cf, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.595232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.608390 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1379/2763] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:20 2022] PUT /identity/v3/projects/7014ddb5f34d4915bb3def14c2814e5f/users/b65ce68ef01646a1a324419da49276cf/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:20.618607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.618908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.619161 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.646797 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] MFA Rules not processed for user `b65ce68ef01646a1a324419da49276cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:20.647923 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:20.710168 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1385/2764] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:20.714602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.733438 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:20.782252 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe937293-7f2d-40a9-be77-8259381a4081 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.786968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.787585 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.787866 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.788141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.788824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.790766 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.820314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1380/2765] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:20.824326 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.835794 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352611e-3a73-4c2a-b1c1-5794dcdd5753 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.905656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.906194 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.906471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.906745 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.907376 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.909348 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.931117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1386/2766] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:20 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:20.935647 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.945773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9149c-9de5-441c-ae51-f229be1984fe admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.951127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.951727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.951998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.952280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.952923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:20.954513 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:20.965605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1381/2767] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:20.969858 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:20.980217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831bdd0-f87a-4e70-9546-c1190f8251ce admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:20.984943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:20.985591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:20.985885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:20.986144 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] PATH_INFO: `/v3/projects/fc71909b91544f2a974f86a10e25ca1a/users/ae6e8e59beb34396be425b8eae464800/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:20.988539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:21.005637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc71909b91544f2a974f86a10e25ca1a, user_id=ae6e8e59beb34396be425b8eae464800, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.008369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.022209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1387/2768] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:20 2022] PUT /identity/v3/projects/fc71909b91544f2a974f86a10e25ca1a/users/ae6e8e59beb34396be425b8eae464800/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:21.026166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.036554 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaaa54-bae7-4c02-84a0-fd945509674c admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.099264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.099852 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.100023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.100341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.101046 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.103019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.114264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1382/2769] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:21.119434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.130143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911411-d085-45f3-9d7f-5c23edd956fd admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.135156 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.135807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.136087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.136395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] PATH_INFO: `/v3/projects/fc71909b91544f2a974f86a10e25ca1a/users/ae6e8e59beb34396be425b8eae464800/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.139318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc71909b91544f2a974f86a10e25ca1a, user_id=ae6e8e59beb34396be425b8eae464800, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.142991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4158b87-e11d-44bd-a2ba-5fe903209628 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.157401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1388/2770] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:21 2022] PUT /identity/v3/projects/fc71909b91544f2a974f86a10e25ca1a/users/ae6e8e59beb34396be425b8eae464800/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:21.166314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.168075 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.168741 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.169067 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.169343 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.180739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516606d-0ab8-4397-9cd9-e2b4020988f8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.202340 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] MFA Rules not processed for user `ae6e8e59beb34396be425b8eae464800`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:21.204580 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.245639 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.247690 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.248285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.248828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] PATH_INFO: `/v3/users/bd7e86dc187e47cfb524e4cf8374bc6a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.250219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7e86dc187e47cfb524e4cf8374bc6a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.252397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.289094 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:21.301791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1389/2771] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:21.310114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.312379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.313440 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7e86dc187e47cfb524e4cf8374bc6a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:21.314119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7e86dc187e47cfb524e4cf8374bc6a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:21.321848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7e86dc187e47cfb524e4cf8374bc6a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:21.324084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7e86dc187e47cfb524e4cf8374bc6a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:21.335907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-9a7ce88a-af89-4bdb-9be5-79a8cc061d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7e86dc187e47cfb524e4cf8374bc6a'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:21.356736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1383/2772] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:21 2022] DELETE /identity/v3/users/bd7e86dc187e47cfb524e4cf8374bc6a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:21.361353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.362011 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.362178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.376650 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8caf9-3130-4fb1-a8b6-b3028c358639 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.382921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:21.384376 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.435654 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.436265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.436557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.436825 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.438670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.451975 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1384/2773] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:21.461304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.462658 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.464842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:21.505216 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.506705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.518109 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1390/2774] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:21.650583 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca4b1c-2884-4068-88bd-56fe12517eea None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.654884 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.655516 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.655796 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.656066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.666402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.667809 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.677645 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1385/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:38:21.938399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.939994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.954276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e892b-5191-466e-9db4-0a26fb3477f1 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.958991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.959624 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.960158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.961415 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:21.962166 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] PATH_INFO: `/v3/users/b65ce68ef01646a1a324419da49276cf` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.962920 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:21.963316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65ce68ef01646a1a324419da49276cf)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.965908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:21.982090 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1424db-586b-4837-9d8a-7253896d564b service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:21.987194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:21.987889 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:21.988171 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:21.988592 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] PATH_INFO: `/v3/projects/0bbc1b0230a949c19ff51a3e9335ac8d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:21.989798 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bbc1b0230a949c19ff51a3e9335ac8d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:21.992355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.018435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65ce68ef01646a1a324419da49276cf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.026807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65ce68ef01646a1a324419da49276cf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.027737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65ce68ef01646a1a324419da49276cf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.035427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65ce68ef01646a1a324419da49276cf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.036238 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3177ac7b-7095-4065-965a-06bb024646d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65ce68ef01646a1a324419da49276cf'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.039042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:22.043651 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbc1b0230a949c19ff51a3e9335ac8d', 'request_id': 'req-27250006-3482-4929-be9f-8186a0e12420'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.055112 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbc1b0230a949c19ff51a3e9335ac8d', 'request_id': 'req-27250006-3482-4929-be9f-8186a0e12420'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.060449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1391/2776] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:21 2022] DELETE /identity/v3/users/b65ce68ef01646a1a324419da49276cf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:22.064617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.064990 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.065351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbc1b0230a949c19ff51a3e9335ac8d', 'request_id': 'req-27250006-3482-4929-be9f-8186a0e12420'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.065471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.076670 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] The token cache is being invalidate because project 0bbc1b0230a949c19ff51a3e9335ac8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:22.076963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.077676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.079602 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1386/2777] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:21 2022] DELETE /identity/v3/projects/0bbc1b0230a949c19ff51a3e9335ac8d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:22.083146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.084696 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.084835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:22.085883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.154079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1392/2778] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:22.161268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27250006-3482-4929-be9f-8186a0e12420 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.163662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.165097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.166089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.166725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.166998 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.167267 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] PATH_INFO: `/v3/users/c9fe561e035a4070ac0da1cf2ae783a3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.168941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fe561e035a4070ac0da1cf2ae783a3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.170898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.202618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:22.221952 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fe561e035a4070ac0da1cf2ae783a3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.222665 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fe561e035a4070ac0da1cf2ae783a3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.228163 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3a297-2775-431d-bac0-1762c00b17b2 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.229825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fe561e035a4070ac0da1cf2ae783a3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.230449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fe561e035a4070ac0da1cf2ae783a3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.233048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.233692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.233982 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.234254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.236179 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.238379 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fe561e035a4070ac0da1cf2ae783a3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.249399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.250813 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.255963 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1387/2779] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:22 2022] DELETE /identity/v3/users/c9fe561e035a4070ac0da1cf2ae783a3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:22.260683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1393/2780] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:22.669103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.670666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.670989 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.672565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.687908 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e021b5-2133-4087-9853-4955e601226e service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.734493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:22.749309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c42-b553-49a9-a4a6-e87665c49222 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.750817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.751453 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.751733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.752009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] PATH_INFO: `/v3/projects/7aabb8efe3014f3bba3d3ab66f65ec95` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.753222 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aabb8efe3014f3bba3d3ab66f65ec95)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.754191 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.754820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.755096 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.755217 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.755367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] PATH_INFO: `/v3/projects/7014ddb5f34d4915bb3def14c2814e5f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.756514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7014ddb5f34d4915bb3def14c2814e5f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.758436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.803526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aabb8efe3014f3bba3d3ab66f65ec95', 'request_id': 'req-2fe969bd-a070-484b-9231-ea4577c4180c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.806441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7014ddb5f34d4915bb3def14c2814e5f', 'request_id': 'req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.809957 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aabb8efe3014f3bba3d3ab66f65ec95', 'request_id': 'req-2fe969bd-a070-484b-9231-ea4577c4180c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.814557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7014ddb5f34d4915bb3def14c2814e5f', 'request_id': 'req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.817765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aabb8efe3014f3bba3d3ab66f65ec95', 'request_id': 'req-2fe969bd-a070-484b-9231-ea4577c4180c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.820892 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7014ddb5f34d4915bb3def14c2814e5f', 'request_id': 'req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.829509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] The token cache is being invalidate because project 7aabb8efe3014f3bba3d3ab66f65ec95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:22.829795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.830424 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.831915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1394/2781] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:22 2022] DELETE /identity/v3/projects/7aabb8efe3014f3bba3d3ab66f65ec95 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:22.832621 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] The token cache is being invalidate because project 7014ddb5f34d4915bb3def14c2814e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:22.832912 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.833562 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.835187 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1388/2782] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:22 2022] DELETE /identity/v3/projects/7014ddb5f34d4915bb3def14c2814e5f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:22.835660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.837086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.838625 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:22.840010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:22.908274 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe033a-b5e6-4c2a-8ef7-9b5bdb7e60f4 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.908966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe969bd-a070-484b-9231-ea4577c4180c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:22.913042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.913557 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:22.913688 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.913931 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.914183 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:22.914295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] PATH_INFO: `/v3/users/ae6e8e59beb34396be425b8eae464800` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.914462 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:22.914721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] PATH_INFO: `/v3/users/1f45c840fcd1428cb5621aa008b51efa` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:22.915337 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6e8e59beb34396be425b8eae464800)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.915881 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f45c840fcd1428cb5621aa008b51efa)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:22.917293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.917962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:22.923815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:22.965514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6e8e59beb34396be425b8eae464800'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.966205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6e8e59beb34396be425b8eae464800'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.970101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f45c840fcd1428cb5621aa008b51efa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.973875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6e8e59beb34396be425b8eae464800'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.974520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6e8e59beb34396be425b8eae464800'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.977800 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f45c840fcd1428cb5621aa008b51efa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.978474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f45c840fcd1428cb5621aa008b51efa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.982992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6e8e59beb34396be425b8eae464800'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.985509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f45c840fcd1428cb5621aa008b51efa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.986214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f45c840fcd1428cb5621aa008b51efa'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:22.996788 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:23.003674 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1389/2783] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:22 2022] DELETE /identity/v3/users/ae6e8e59beb34396be425b8eae464800 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:23.005307 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1395/2784] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:22 2022] DELETE /identity/v3/users/1f45c840fcd1428cb5621aa008b51efa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:23.378215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.379739 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.386388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.389703 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.483899 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ec44-3800-44d9-bb3d-38b49218d189 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.488948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.489713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.490095 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.490544 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] PATH_INFO: `/v3/projects/fc71909b91544f2a974f86a10e25ca1a` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.492199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc71909b91544f2a974f86a10e25ca1a)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.494010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.497079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a620783-9539-427d-8cbb-e2ffe0f93350 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.505299 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.506449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.507026 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.507465 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] PATH_INFO: `/v3/projects/22965960fb084c0190a880dceeec13d3` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.508671 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22965960fb084c0190a880dceeec13d3)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.510555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.567380 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc71909b91544f2a974f86a10e25ca1a', 'request_id': 'req-4e7174dd-8fd9-475d-8569-a6a08cc62e09'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.572801 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22965960fb084c0190a880dceeec13d3', 'request_id': 'req-7f6918cf-0cfa-4451-9131-3d2e82075cb1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.581432 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22965960fb084c0190a880dceeec13d3', 'request_id': 'req-7f6918cf-0cfa-4451-9131-3d2e82075cb1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.590770 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22965960fb084c0190a880dceeec13d3', 'request_id': 'req-7f6918cf-0cfa-4451-9131-3d2e82075cb1'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.594246 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc71909b91544f2a974f86a10e25ca1a', 'request_id': 'req-4e7174dd-8fd9-475d-8569-a6a08cc62e09'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.604261 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc71909b91544f2a974f86a10e25ca1a', 'request_id': 'req-4e7174dd-8fd9-475d-8569-a6a08cc62e09'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.607569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] The token cache is being invalidate because project 22965960fb084c0190a880dceeec13d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:23.607817 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.608684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-7f6918cf-0cfa-4451-9131-3d2e82075cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.610502 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1396/2785] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:23 2022] DELETE /identity/v3/projects/22965960fb084c0190a880dceeec13d3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:23.615123 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cc42-b606-408b-b292-d7a22383f09f None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.615601 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cc42-b606-408b-b292-d7a22383f09f None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.615961 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cc42-b606-408b-b292-d7a22383f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.623726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] The token cache is being invalidate because project fc71909b91544f2a974f86a10e25ca1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:23.624104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.624860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e7174dd-8fd9-475d-8569-a6a08cc62e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:23.626514 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1390/2786] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:23 2022] DELETE /identity/v3/projects/fc71909b91544f2a974f86a10e25ca1a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:23.631159 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f280a-7ba9-44e8-b00c-2aea4b652db1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.631543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f280a-7ba9-44e8-b00c-2aea4b652db1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.632038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f280a-7ba9-44e8-b00c-2aea4b652db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.642126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3b72cc42-b606-408b-b292-d7a22383f09f None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:23.643402 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3b72cc42-b606-408b-b292-d7a22383f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.654327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9d6f280a-7ba9-44e8-b00c-2aea4b652db1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:23.655226 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:23.656033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9d6f280a-7ba9-44e8-b00c-2aea4b652db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.711314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1397/2787] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:23.719720 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.720005 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.720227 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.727946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1391/2788] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:23.736134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.736768 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.737219 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.741403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:23.742698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.753733 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1398/2789] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:23.757814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.760208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:23.761509 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.769287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3340-1c57-4fd9-9385-9bbc22a9ab4c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.770704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1392/2790] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:23.774599 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.778948 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.779798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.780072 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.780548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.781195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:23.781495 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.783090 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.789154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f365cd0-a605-4c08-9dd5-2d670f6b2ed8 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.798564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1399/2791] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:38:23.798725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.798725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.798725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.798725 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.799163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:23.799163 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.800571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.804903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.816198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1393/2792] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:38:23.816198 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.816533 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f4eff-3b59-4d2f-86ad-4ffc268755cb admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.816640 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.819592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.820181 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.820484 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.820754 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.821381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.823239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.830067 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:23.854756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1400/2793] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:23.854756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:23.883830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32dfb2-c15b-488e-8409-22556994c258 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.916356 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d203d0-e525-4e52-b3c5-593e1f086ca7 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:23.921225 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.921904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.922180 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.922455 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.923077 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:23.924995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:23.934342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:23.935097 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:23.935799 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:23.936078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:23.938349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:23.954174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1401/2794] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:23.954966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.009127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.010536 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.019063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d92c-549f-4f85-89a9-b8a83019816a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.021932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1394/2795] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:24.022495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.026781 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.027617 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.027894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.028154 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.029441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.031673 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.034704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f475519-8167-4fb8-ab91-67311fc31a54 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.037944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.038551 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.038832 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.039103 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.039986 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.040692 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:24.041960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.059682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1402/2796] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:23 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:24.063780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.070805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1395/2797] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:24.070805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.075561 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4a7a3-ff62-4852-8fb6-c0f5227e424d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.083006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f03ad-0512-4fca-9873-7e8bbd032af3 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.084950 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.085609 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.086032 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.086317 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.086965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.089017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.092403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.093019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.093301 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.093801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.094240 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.096203 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.103204 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1403/2798] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.107693 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.109976 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1396/2799] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.114030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.118374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ade13-7e55-43d1-bdd3-238ba1ee0ecd admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.125236 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.125918 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.126266 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.126404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801210-7160-40e2-9414-ec807fdd0260 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.126660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.127430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.129126 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.131024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.131704 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.131978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.132278 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.132908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.134473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.141564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1404/2800] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.147064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.151243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1397/2801] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.155578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.159276 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d2250-7416-418b-b491-3b8ed08a734a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.165341 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359db3-4159-497d-97ae-58b5db1833ad admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.165807 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.166471 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.166748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.167014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] PATH_INFO: `/v3/projects/741fd985226b460a93af30e55719aab2/users/e40ff402e4f74abaae63a8230885827b/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.171273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.171941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.172222 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.172529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] PATH_INFO: `/v3/projects/0c8167929fcc48ea9c50c9832a8fb045/users/444de4c1a1444d88bf96bca7cee73654/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.185158 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741fd985226b460a93af30e55719aab2, user_id=e40ff402e4f74abaae63a8230885827b, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.188143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868e2a9-d608-49d5-96ee-f5e5eb9edff8 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.189878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8167929fcc48ea9c50c9832a8fb045, user_id=444de4c1a1444d88bf96bca7cee73654, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.193378 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3a493-5518-4dbf-8c0c-d9911c1fc8d5 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.204639 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1405/2802] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:24 2022] PUT /identity/v3/projects/741fd985226b460a93af30e55719aab2/users/e40ff402e4f74abaae63a8230885827b/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:24.208268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1398/2803] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:24 2022] PUT /identity/v3/projects/0c8167929fcc48ea9c50c9832a8fb045/users/444de4c1a1444d88bf96bca7cee73654/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:24.209852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.210303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.210603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.219731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.220049 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.220303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.232440 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:24.233672 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:24.249891 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:24.255765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] MFA Rules not processed for user `444de4c1a1444d88bf96bca7cee73654`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:24.257427 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:24.304885 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1406/2804] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:24.315850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.333790 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1399/2805] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:24.338691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.351190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88920eb4-e64c-4614-a201-dcf0536aa948 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.356797 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.357539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.357825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.358133 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.358775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.360849 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.377147 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1fe2c-e22e-40e3-8bb6-5807d7792643 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.381925 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.382750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.383558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.383558 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.394282 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.395534 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1400/2806] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:24.395993 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.399106 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.406612 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1407/2807] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:38:24.468510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332c617-c56e-4f9a-996b-c67437c61ee1 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.474491 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.475143 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.475433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.475984 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.476405 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.478683 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.506618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1401/2808] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:24.507809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.520666 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8604f-000a-4ced-b7f8-4664e998fc05 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.525941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.526582 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.526862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.528230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.528638 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.530273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.541313 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1408/2809] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.545606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.557969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2abad-e4ed-46f5-a0bd-27cac7c33ece admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.563662 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.564368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.564789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.565138 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] PATH_INFO: `/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615/users/0d2505ccba1e43268ce159850569f7f8/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.582820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c3c93fa0a74015b3cbccd75f9e8615, user_id=0d2505ccba1e43268ce159850569f7f8, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.585546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.601769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1402/2810] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:24 2022] PUT /identity/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615/users/0d2505ccba1e43268ce159850569f7f8/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:24.606427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.620644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada62c-346a-4087-b9a6-41d155717ff0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.673927 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:24.709773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.710401 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.710668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.710934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.711606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.713213 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e97dd1-df1b-4078-90c5-c8276a4bdb16 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.725603 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1409/2811] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:24.729586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.743294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb2c30-46d4-44f1-9a25-d71bd23a1637 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:24.770969 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:24.772368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.772784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.773104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] PATH_INFO: `/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615/users/0d2505ccba1e43268ce159850569f7f8/roles/e0af1611fa564e7d8d26e3e289f6c5c5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.776758 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c3c93fa0a74015b3cbccd75f9e8615, user_id=0d2505ccba1e43268ce159850569f7f8, role_id=e0af1611fa564e7d8d26e3e289f6c5c5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:24.785304 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:24.796978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1403/2812] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:24 2022] PUT /identity/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615/users/0d2505ccba1e43268ce159850569f7f8/roles/e0af1611fa564e7d8d26e3e289f6c5c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:24.807433 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:24.807771 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:24.808011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:24.852434 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] MFA Rules not processed for user `0d2505ccba1e43268ce159850569f7f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:24.853812 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:24.958883 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1410/2813] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:24.974319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:24.976556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:25.018895 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:25.049247 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5f40-33f2-4114-a091-900dab97e899 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:25.053473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:25.054365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:25.054853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:25.055273 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:25.071081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:25.073115 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab79a3-5714-4ffd-83e0-0c09d9c0c063 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:25.092553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1404/2814] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:24 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:25.159999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:25.162555 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:25.175719 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4d29a-f630-4e95-a8a9-6d496cac05a7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:25.179652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:25.180126 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b22d-1dc5-42bf-9843-645c48eecf46 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:25.180295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:25.180547 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:25.180821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:25.181303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b22d-1dc5-42bf-9843-645c48eecf46 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:25.182019 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b22d-1dc5-42bf-9843-645c48eecf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:25.220693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-73a0b22d-1dc5-42bf-9843-645c48eecf46 None None] MFA Rules not processed for user `e40ff402e4f74abaae63a8230885827b`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:25.224565 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-73a0b22d-1dc5-42bf-9843-645c48eecf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:25.259413 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:25.260889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:25.285024 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1411/2815] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:25.310140 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1405/2816] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:25.323318 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:25.334384 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75f8ae-3e8d-4574-9158-fb56171dedb1 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:25.339618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:25.341725 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:25.342185 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:25.342637 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:25.354886 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:25.357497 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:25.372971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1412/2817] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:26.882978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5084-51ec-4051-ac3c-c5268227644a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:26.883632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5084-51ec-4051-ac3c-c5268227644a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:26.883632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5084-51ec-4051-ac3c-c5268227644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:26.909629 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-547b5084-51ec-4051-ac3c-c5268227644a None None] MFA Rules not processed for user `e40ff402e4f74abaae63a8230885827b`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:26.911275 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-547b5084-51ec-4051-ac3c-c5268227644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:26.920338 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1406/2818] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:29.143952 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:29.203253 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:29.228552 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9c2a-b610-4d9c-839a-483e110ded25 service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:29.233423 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:29.234051 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:29.234324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:29.234584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] PATH_INFO: `/v3/users/444de4c1a1444d88bf96bca7cee73654` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:29.235712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444de4c1a1444d88bf96bca7cee73654)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:29.237658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:29.284500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444de4c1a1444d88bf96bca7cee73654'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.291792 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444de4c1a1444d88bf96bca7cee73654'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.292437 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444de4c1a1444d88bf96bca7cee73654'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.300027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444de4c1a1444d88bf96bca7cee73654'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.300740 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444de4c1a1444d88bf96bca7cee73654'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.319113 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1413/2819] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:29 2022] DELETE /identity/v3/users/444de4c1a1444d88bf96bca7cee73654 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:29.323789 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:29.324910 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:29.325181 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:29.345423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:29.346637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:29.406423 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1407/2820] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:29.416189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:29.417596 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:29.481296 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1719622-25c5-4e58-a4e1-dd121fd8dc10 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:29.485232 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:29.485846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:29.486141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:29.486387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:29.496707 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:29.498120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:29.507839 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1414/2821] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:29.771940 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:29.773445 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:29.797010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365255c-fe31-46f6-ba29-194e21a014ec None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:29.803250 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:29.804124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:29.804473 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:29.804863 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] PATH_INFO: `/v3/projects/0c8167929fcc48ea9c50c9832a8fb045` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:29.806205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8167929fcc48ea9c50c9832a8fb045)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:29.808208 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:29.861807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8167929fcc48ea9c50c9832a8fb045', 'request_id': 'req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.866345 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:29.872755 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8167929fcc48ea9c50c9832a8fb045', 'request_id': 'req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.881590 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8167929fcc48ea9c50c9832a8fb045', 'request_id': 'req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.896013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] The token cache is being invalidate because project 0c8167929fcc48ea9c50c9832a8fb045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:29.896331 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.896985 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:29.898495 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1408/2822] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:29 2022] DELETE /identity/v3/projects/0c8167929fcc48ea9c50c9832a8fb045 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:29.903174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:29.913131 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9d06a-ed12-4a33-bd39-a368db46433f service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:29.971735 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:29.972442 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:29.972815 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:29.973101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] PATH_INFO: `/v3/users/0d2505ccba1e43268ce159850569f7f8` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:29.974280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2505ccba1e43268ce159850569f7f8)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:29.976272 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:29.981273 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:30.028924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2505ccba1e43268ce159850569f7f8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.036482 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2505ccba1e43268ce159850569f7f8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.037140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2505ccba1e43268ce159850569f7f8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.044869 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2505ccba1e43268ce159850569f7f8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.045607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-07c6dd22-f30a-407c-8ac3-00e392acc476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2505ccba1e43268ce159850569f7f8'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.063659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1415/2823] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:29 2022] DELETE /identity/v3/users/0d2505ccba1e43268ce159850569f7f8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:38:30.544618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:30.546098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:30.560101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deeb60-7c5c-4461-8f83-3653a2ebd34e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:30.627810 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:30.628490 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:30.628769 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:30.629038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] PATH_INFO: `/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:30.630162 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c3c93fa0a74015b3cbccd75f9e8615)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:30.632057 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:30.680900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c3c93fa0a74015b3cbccd75f9e8615', 'request_id': 'req-af7fc750-f12e-47e1-a00b-4c71af90f4c4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.688017 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c3c93fa0a74015b3cbccd75f9e8615', 'request_id': 'req-af7fc750-f12e-47e1-a00b-4c71af90f4c4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.706443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c3c93fa0a74015b3cbccd75f9e8615', 'request_id': 'req-af7fc750-f12e-47e1-a00b-4c71af90f4c4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.727289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] The token cache is being invalidate because project c1c3c93fa0a74015b3cbccd75f9e8615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:30.727819 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.728660 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-af7fc750-f12e-47e1-a00b-4c71af90f4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:30.730468 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1409/2824] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:30 2022] DELETE /identity/v3/projects/c1c3c93fa0a74015b3cbccd75f9e8615 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:30.735439 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:30.735973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:30.736076 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:30.759124 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:30.760369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:30.820002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1416/2825] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:30.828667 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:30.828966 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:30.829239 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:30.850520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:30.852225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:30.860992 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1410/2826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:30.865230 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:30.874873 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fa9cf-abd6-4e06-b269-e30a080ef064 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:30.879882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:30.880517 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:30.880789 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:30.881081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:30.881726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:30.882027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:30.883613 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:30.895795 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1417/2827] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:30.900110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:30.910127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9065fc-c9b9-408a-9f77-ae6d3f392e11 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:30.914967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:30.915563 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:30.915840 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:30.916110 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:30.916767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:30.919408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:30.952189 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1411/2828] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:30 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:30.953652 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:30.963772 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5cae-a829-4dce-82dc-1483ac6cf41d admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:31.020417 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:31.021023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.021300 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.021570 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.022218 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:31.024167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:31.040040 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:31.046791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1418/2829] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:30 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:31.050523 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:31.061744 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399598-f3d5-4bba-9e04-0e310e2ed5d9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:31.066828 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:31.067526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.067834 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.068109 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.068826 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:31.070424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:31.081251 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1412/2830] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:31.085151 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:31.095864 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f910fdf-8cda-4d34-b93f-558fbab00c2a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:31.107916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:31.108569 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.108848 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.109117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.109761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:31.111337 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd82d4-0bc2-4060-80c4-4fa14d286873 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:31.123563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1419/2831] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:31.127767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:31.137794 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2427b6-5543-40af-9ab4-630ab5b3a3ab admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:31.142486 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:31.143128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.143402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.143666 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] PATH_INFO: `/v3/projects/ffd8ceda1d624fb787751c5d927eafa9/users/d9c7326d223e40eb979abde61a178557/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.159616 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd8ceda1d624fb787751c5d927eafa9, user_id=d9c7326d223e40eb979abde61a178557, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:31.162258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:31.176298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1413/2832] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:31 2022] PUT /identity/v3/projects/ffd8ceda1d624fb787751c5d927eafa9/users/d9c7326d223e40eb979abde61a178557/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:31.188069 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786b48e-6556-4633-942c-5701bb191b5d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.188400 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786b48e-6556-4633-942c-5701bb191b5d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.188670 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786b48e-6556-4633-942c-5701bb191b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.215668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a786b48e-6556-4633-942c-5701bb191b5d None None] MFA Rules not processed for user `d9c7326d223e40eb979abde61a178557`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:31.216890 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a786b48e-6556-4633-942c-5701bb191b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:31.280172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1420/2833] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:31.291027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:31.293396 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:31.304231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:31.352626 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd79b7-3237-4fe0-bf11-ad8fa3791b06 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:31.356942 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:31.357543 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:31.357820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:31.358091 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:31.366489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:31.367918 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:31.377124 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1414/2834] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:41.277347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=37 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:41.372864 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=22 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:41.746761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ba17c-319a-4025-8105-1d1925f7cd19 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:41.747041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ba17c-319a-4025-8105-1d1925f7cd19 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:41.747316 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ba17c-319a-4025-8105-1d1925f7cd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:41.767416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-033ba17c-319a-4025-8105-1d1925f7cd19 None None] MFA Rules not processed for user `e40ff402e4f74abaae63a8230885827b`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:41.768658 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-033ba17c-319a-4025-8105-1d1925f7cd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:41.824412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1421/2835] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:49.114958 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:49.182033 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806464cf-e953-4cd6-8587-f77738dbd67e service glance] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:49.186630 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:49.187256 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.187529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.187800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] PATH_INFO: `/v3/users/d9c7326d223e40eb979abde61a178557` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.188961 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c7326d223e40eb979abde61a178557)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:49.190851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:49.242946 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c7326d223e40eb979abde61a178557'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.243632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c7326d223e40eb979abde61a178557'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.251289 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c7326d223e40eb979abde61a178557'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.251908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c7326d223e40eb979abde61a178557'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.258997 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c7326d223e40eb979abde61a178557'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.277184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1415/2836] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:38:49 2022] DELETE /identity/v3/users/d9c7326d223e40eb979abde61a178557 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:49.282357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.282852 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.282955 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.301835 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:49.303009 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.360786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1422/2837] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:49.370540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:49.372793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.428349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ae3d3-ccd2-48af-b64b-29fdcb49f075 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:49.431949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:49.432575 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.432853 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.433123 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.442408 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:49.443827 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239145-79d1-4123-aaa2-c66fffecb22c service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:49.453801 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1416/2838] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:49.721882 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:49.723324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.735492 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eb1f5-69bf-4459-b407-9fefbd190bf7 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:49.740023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:49.741436 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.741710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.742149 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] PATH_INFO: `/v3/projects/ffd8ceda1d624fb787751c5d927eafa9` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.743396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd8ceda1d624fb787751c5d927eafa9)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:49.745863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:49.794409 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd8ceda1d624fb787751c5d927eafa9', 'request_id': 'req-8a07e538-1397-431b-811e-ac201396de89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.800579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd8ceda1d624fb787751c5d927eafa9', 'request_id': 'req-8a07e538-1397-431b-811e-ac201396de89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.808445 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd8ceda1d624fb787751c5d927eafa9', 'request_id': 'req-8a07e538-1397-431b-811e-ac201396de89'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.819125 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] The token cache is being invalidate because project ffd8ceda1d624fb787751c5d927eafa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:38:49.819405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.820027 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-8a07e538-1397-431b-811e-ac201396de89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:38:49.821516 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1423/2839] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:38:49 2022] DELETE /identity/v3/projects/ffd8ceda1d624fb787751c5d927eafa9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:49.826604 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.826886 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.827156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.846708 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:49.847880 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.865354 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:49.912206 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1417/2840] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:49.920489 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.920757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.921028 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.941064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:49.942245 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.950285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1424/2841] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:49.954367 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:49.963843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5647538-ff46-409c-92ae-11639bffc131 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:49.968528 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:49.969120 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.969395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.969917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.970550 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:38:49.970851 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:49.972453 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:49.984313 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1418/2842] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:38:49.988360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:49.998189 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712a693-6514-4ddc-ac2a-44592e27909d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.002786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.003364 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.003635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.003901 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.004554 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.006520 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.028083 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:50.038328 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1425/2843] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:38:49 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:38:50.039387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:50.049825 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790460d0-c9ea-4640-85ad-66a3540211b8 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.107296 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.107934 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.108216 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.108520 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.109165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.111084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.131973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1419/2844] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:38:50 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:38:50.136055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:50.146421 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769c19a-a007-4ab2-81de-eb5f1d4ac202 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.151084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.151698 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.151975 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.152959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.153863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.155472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.165334 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1426/2845] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:50.169979 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:50.179698 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a3ab-acf6-49b9-bcc6-277a8bc23e0e admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.184644 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.185232 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.185529 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.185786 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.186412 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.188021 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb73c5-8f9a-4f76-966f-1d07ab4e2ee2 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.198526 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1420/2846] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:38:50.202435 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:50.212142 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd979d5-0513-4efc-aac4-31b9c4d3c71c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.217231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.217863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.218138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.218405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] PATH_INFO: `/v3/projects/e82c6ec8b6954e6bbf091dde0e889d5d/users/9b622ac579f54191bfb01a6ece1a75f3/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.233461 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82c6ec8b6954e6bbf091dde0e889d5d, user_id=9b622ac579f54191bfb01a6ece1a75f3, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.236050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.248704 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1427/2847] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:38:50 2022] PUT /identity/v3/projects/e82c6ec8b6954e6bbf091dde0e889d5d/users/9b622ac579f54191bfb01a6ece1a75f3/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:38:50.260086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.260397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.260668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.287869 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] MFA Rules not processed for user `9b622ac579f54191bfb01a6ece1a75f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:50.289098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:50.358730 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1421/2848] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:38:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:50.369644 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:50.371623 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:50.429846 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6714dbe-5bae-44bf-afc3-7c1a2f43f808 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:50.433428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:50.434057 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:50.434430 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:50.434712 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:50.443031 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:50.444532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f98ea6-2790-4386-85b2-6593a0415384 service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:50.454188 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1428/2849] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:00.354806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:00.449700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:08.879727 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:08.881682 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:08.940404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a50a6-8a3b-49ae-addc-b40ac529d7e0 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:08.944012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:08.944650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:08.944925 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:08.945192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:08.946916 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:09.028609 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:09.030006 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:09.049745 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1422/2850] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:10.198449 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:10.198857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:10.198953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:10.219369 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:10.220660 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:10.245781 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:10.257245 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc5ebf-f377-4912-903d-e00437228a81 service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:10.261330 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:10.261935 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:10.262262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:10.262539 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:10.264549 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:10.280881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:10.282303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:10.282903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1429/2851] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:39:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:10.296280 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1423/2852] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:39:10.330977 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:10.332410 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:10.411203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9adfc-c7df-443c-a579-1d84c23c3ff5 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:10.417084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:10.417722 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:10.418114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:10.418256 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:10.426904 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:10.428669 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635e0da-216e-4b24-bb05-bb8990c113f6 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:10.438406 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1430/2853] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:39:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:39:12.509215 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:12.592319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f5c26-b90b-472a-adaa-05146466ceed service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:12.597471 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:12.598113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:12.598400 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:12.598669 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] PATH_INFO: `/v3/users/9b622ac579f54191bfb01a6ece1a75f3` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:12.599951 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b622ac579f54191bfb01a6ece1a75f3)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:12.602211 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:12.615042 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:12.666860 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b622ac579f54191bfb01a6ece1a75f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:12.667839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b622ac579f54191bfb01a6ece1a75f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:12.682505 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b622ac579f54191bfb01a6ece1a75f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:12.683496 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b622ac579f54191bfb01a6ece1a75f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:12.692559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b622ac579f54191bfb01a6ece1a75f3'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:12.713277 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1424/2854] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:39:12 2022] DELETE /identity/v3/users/9b622ac579f54191bfb01a6ece1a75f3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:12.717926 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:12.718468 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:12.719114 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:12.741105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:12.742862 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:12.820283 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1431/2855] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:39:12.828879 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:12.830580 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:12.924435 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82065b-0f9e-411e-99a5-1a3e168644ab admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:12.926284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:12.926284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:12.926284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:12.926284 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:12.939105 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:12.940741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0562bd0-3616-4a01-b4c8-2a1b5a416c6d service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:12.952914 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1425/2856] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:39:13.298130 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.300087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:13.317173 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725849f-2530-4f31-ba87-55cab67a9603 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.335611 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.336285 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.336571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.336840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] PATH_INFO: `/v3/projects/e82c6ec8b6954e6bbf091dde0e889d5d` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.337996 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82c6ec8b6954e6bbf091dde0e889d5d)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.340078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.409002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:13.423783 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82c6ec8b6954e6bbf091dde0e889d5d', 'request_id': 'req-01610d00-92f3-47a0-984c-60c3b5392388'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:13.430258 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82c6ec8b6954e6bbf091dde0e889d5d', 'request_id': 'req-01610d00-92f3-47a0-984c-60c3b5392388'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:13.438953 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82c6ec8b6954e6bbf091dde0e889d5d', 'request_id': 'req-01610d00-92f3-47a0-984c-60c3b5392388'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:13.452231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] The token cache is being invalidate because project e82c6ec8b6954e6bbf091dde0e889d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:39:13.452792 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:13.453422 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-01610d00-92f3-47a0-984c-60c3b5392388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:13.455219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1432/2857] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:39:13 2022] DELETE /identity/v3/projects/e82c6ec8b6954e6bbf091dde0e889d5d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:13.459949 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.460237 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.460726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.484027 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:13.485291 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:13.564421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1426/2858] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:13.567030 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.567324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.567765 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.590345 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:13.592231 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:13.601473 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1433/2859] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:13.605365 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.616775 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed01e4-80d5-442c-85f1-29a5c1e9dc29 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.621737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.622347 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.622624 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.622895 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.624168 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:39:13.624632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.626370 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.639221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1427/2860] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:39:13.644485 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.654973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eed87-f3dc-4080-85f2-cde38f887cf1 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.660490 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.661068 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.661357 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.661628 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.662302 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.664481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.710065 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1434/2861] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:39:13 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:39:13.715010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.728086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf712a0f-32b2-42f0-b3a8-8404ef156fd0 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.746909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:13.807833 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.808668 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.809013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.809355 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.810089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.812719 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.840298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1428/2862] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:39:13 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:39:13.844360 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.855052 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5deb854-ab43-4730-870e-b9137991ae79 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.860602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.861209 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.861487 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.861753 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.862399 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.864195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.878039 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1435/2863] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:39:13.882488 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.899978 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ba24-51f8-4836-a6bd-3f0363ec8c75 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.905302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.906295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.907149 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.907606 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.908333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.912190 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b7196-e2f2-4d01-b128-e2f7b349891a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:13.929932 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1429/2864] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:39:13.931254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:13.944101 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b2df0-f95b-4267-afce-7071f98ee0d0 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:13.951477 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:13.952297 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:13.952737 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:13.953117 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] PATH_INFO: `/v3/projects/0435e36fe1ad4d99a3f6b076e469469e/users/ab2976ba8a5f417087c128c860474d4d/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:13.974761 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0435e36fe1ad4d99a3f6b076e469469e, user_id=ab2976ba8a5f417087c128c860474d4d, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:13.978576 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:14.004405 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1436/2865] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:39:13 2022] PUT /identity/v3/projects/0435e36fe1ad4d99a3f6b076e469469e/users/ab2976ba8a5f417087c128c860474d4d/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:14.008547 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415fbc-edde-4354-a6c9-6de13dc42b31 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:14.009351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415fbc-edde-4354-a6c9-6de13dc42b31 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:14.009735 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415fbc-edde-4354-a6c9-6de13dc42b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:14.051865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-79415fbc-edde-4354-a6c9-6de13dc42b31 None None] MFA Rules not processed for user `ab2976ba8a5f417087c128c860474d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:14.053517 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-79415fbc-edde-4354-a6c9-6de13dc42b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:14.149566 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1430/2866] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:14.163246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:14.166377 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:14.234747 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081735b1-0ccc-4407-8df0-048667b5b713 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:14.238635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:14.239219 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:14.239508 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:14.239936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:14.250941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:14.252470 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:14.262537 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1437/2867] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:24.146351 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:24.258379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=70,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:28.620111 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:28.620633 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:28.620738 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:28.668868 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:28.670064 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:28.735350 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1431/2868] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:39:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:28.747531 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:28.749821 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:28.847647 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df65753-6329-4506-8326-4013fa5c2e0e service cinder] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:28.852780 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:28.853385 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:28.853662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:28.853931 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:28.863509 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:28.865120 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:28.873370 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1438/2869] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:39:30.838089 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:30.839540 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:30.906231 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b43fa-01ca-4813-b662-8ccd4049111a None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:30.910677 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:30.911303 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:30.911573 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:30.911839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] PATH_INFO: `/v3/users/e40ff402e4f74abaae63a8230885827b` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:30.912956 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40ff402e4f74abaae63a8230885827b)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:30.914862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:30.961327 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40ff402e4f74abaae63a8230885827b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:30.961903 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40ff402e4f74abaae63a8230885827b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:30.968650 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40ff402e4f74abaae63a8230885827b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:30.969268 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40ff402e4f74abaae63a8230885827b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:30.976421 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-99ded5e6-5b0f-43cb-8a46-e2442b81e57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40ff402e4f74abaae63a8230885827b'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:30.984577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:30.996081 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1432/2870] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:39:30 2022] DELETE /identity/v3/users/e40ff402e4f74abaae63a8230885827b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:31.168205 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.179575 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28eae4-a6e8-4d72-933d-74ecf2531fcc service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.241708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.242309 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.242607 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.242902 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] PATH_INFO: `/v3/projects/741fd985226b460a93af30e55719aab2` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.244078 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741fd985226b460a93af30e55719aab2)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.246132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.293777 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741fd985226b460a93af30e55719aab2', 'request_id': 'req-dd0baa2f-3094-4687-8feb-0319b79767c6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:31.300633 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741fd985226b460a93af30e55719aab2', 'request_id': 'req-dd0baa2f-3094-4687-8feb-0319b79767c6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:31.308201 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741fd985226b460a93af30e55719aab2', 'request_id': 'req-dd0baa2f-3094-4687-8feb-0319b79767c6'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:31.315143 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:31.319786 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] The token cache is being invalidate because project 741fd985226b460a93af30e55719aab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:39:31.320280 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:31.321037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-dd0baa2f-3094-4687-8feb-0319b79767c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:31.322936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1439/2871] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:39:31 2022] DELETE /identity/v3/projects/741fd985226b460a93af30e55719aab2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:31.327904 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.328192 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.328504 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.348548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:31.349741 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:31.408116 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1433/2872] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:31.416374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.416659 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.416934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.437132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:31.438312 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:31.446035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1440/2873] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:31.450220 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.460084 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d607db-507f-4b5f-a3af-7819b98813b1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.464564 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.465151 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.465424 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.465693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] PATH_INFO: `/v3/domains` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.466322 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:39:31.466619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.468820 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.480302 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1434/2874] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:39:31.485229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.494429 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c60d2-cb16-4a00-b8e2-d056408db8be None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.499202 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.499791 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.500081 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.500381 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.501015 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.502891 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.529107 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1441/2875] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:39:31 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:39:31.533793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.543401 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ece216-4281-4bb0-8304-8ad29c7e8901 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.598527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.599127 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.599406 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.599680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] PATH_INFO: `/v3/users` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.600323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.602224 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.625258 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1435/2876] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:39:31 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:39:31.632097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.640138 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68151175-efe1-4b42-8f3a-6950fc5a9f4a admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.645079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.645681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.645960 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.646233 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.646874 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.648500 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.659585 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1442/2877] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:39:31.663695 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.674460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae81717-2dd9-4e0d-ab82-2d5dd26d8204 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.679559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.680035 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.680387 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.683700 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.684409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.686254 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00552ca-30be-44ec-ab64-5832c87a854d admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.698065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1436/2878] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:39:31.702744 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.713097 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca021443-a32c-4af2-af8f-12d6b0ce6ac4 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.718532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.719254 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.719538 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.720086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] PATH_INFO: `/v3/projects/48392d941b474df7b1b7e26b0c22e069/users/a9aebf7f15f04ad39089fe580c014c88/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.734973 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48392d941b474df7b1b7e26b0c22e069, user_id=a9aebf7f15f04ad39089fe580c014c88, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.737701 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.750605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1443/2879] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:39:31 2022] PUT /identity/v3/projects/48392d941b474df7b1b7e26b0c22e069/users/a9aebf7f15f04ad39089fe580c014c88/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:31.760844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.761128 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.761403 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.789260 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] MFA Rules not processed for user `a9aebf7f15f04ad39089fe580c014c88`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:31.790463 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:31.835342 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:31.866325 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1437/2880] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:31.878884 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:31.880705 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:31.943167 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf22fc-ef23-49b8-91b9-259bd73d7c4c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:31.946798 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0da5e4ab96bb4a9584682f32cbafb2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:31.947394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:31.947668 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:31.947934 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:31.956605 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:31.957998 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0906a2-d741-4231-9921-450ecab3b16b service glance] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:31.966571 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1444/2881] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:33.874199 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:33.875778 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:33.943770 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af020c-2171-465a-850d-89f4e9ba01a1 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:33.948214 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:33.948862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:33.949136 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:33.949404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] PATH_INFO: `/v3/users/ab2976ba8a5f417087c128c860474d4d` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:33.950584 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2976ba8a5f417087c128c860474d4d)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:33.952686 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:34.001125 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2976ba8a5f417087c128c860474d4d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.001767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2976ba8a5f417087c128c860474d4d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.009030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2976ba8a5f417087c128c860474d4d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.009680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2976ba8a5f417087c128c860474d4d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.016907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2976ba8a5f417087c128c860474d4d'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.034765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1438/2882] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:39:33 2022] DELETE /identity/v3/users/ab2976ba8a5f417087c128c860474d4d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:34.039228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:34.039723 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:34.039861 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:34.060239 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:34.061743 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:34.123174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1445/2883] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:34.133317 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:34.134742 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:34.193619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bc5f6-7806-4e14-acc8-71a133107563 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:34.197404 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:34.198024 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:34.198319 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:34.198591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:34.208545 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:34.209970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:34.220368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1439/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:34.487995 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:34.489648 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:34.506237 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7961ee-0180-4109-b93d-e23464f54e4d None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:34.511635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:34.512416 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:34.512757 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:34.513033 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] PATH_INFO: `/v3/projects/0435e36fe1ad4d99a3f6b076e469469e` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:34.515190 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0435e36fe1ad4d99a3f6b076e469469e)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:34.517085 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:34.556374 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:34.570152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0435e36fe1ad4d99a3f6b076e469469e', 'request_id': 'req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.576483 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0435e36fe1ad4d99a3f6b076e469469e', 'request_id': 'req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.584602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0435e36fe1ad4d99a3f6b076e469469e', 'request_id': 'req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.595889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] The token cache is being invalidate because project 0435e36fe1ad4d99a3f6b076e469469e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:39:34.596310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.596809 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-c14abe2b-a221-43e0-9c68-b8e2f35d67cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:39:34.598355 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1446/2885] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:39:34 2022] DELETE /identity/v3/projects/0435e36fe1ad4d99a3f6b076e469469e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:39:43.041635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:43.043044 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:43.054255 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:43.102960 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037df40-991d-4dc0-a7b1-54a884b7bdfe service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:43.150349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:43.150943 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:43.151243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:43.151510 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:43.153223 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:43.210524 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:43.211988 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:43.221893 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1440/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:44.591936 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:51.097665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:51.098315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:51.098315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:51.117717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] MFA Rules not processed for user `57daecc890ac421b8eb7b5fb4bf25411`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:39:51.118928 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:51.125726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1447/2887] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:39:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:39:51.477767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:51.479166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:51.535793 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374fb6d-803e-43fe-a4fc-ee31913453c6 service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:51.539527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:51.540131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:51.540442 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:51.540715 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:51.543023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:51.554034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:51.555460 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:51.563876 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1441/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:52.075023 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:52.076823 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:52.157195 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948fdf5-f5f8-41e2-b99c-195f4ce28b90 None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:52.163782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:52.164641 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:52.164916 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:52.165208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:52.166959 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:52.179943 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:52.181539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:52.191764 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1448/2889] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:56.068265 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:39:56.069808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:56.080272 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b5fc9-1a3a-4da1-baad-3636dd9be78f service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:39:56.084225 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:39:56.084839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:39:56.085117 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:39:56.085386 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:39:56.087113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:39:56.097134 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:39:56.098571 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646bf7-2cc6-4d3e-a042-70e292ce9528 service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:39:56.106763 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1442/2890] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:39:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:01.894574 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:01.895969 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:01.959387 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512aa4-893d-4e26-81a5-43807c03d6d3 service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:01.963889 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:01.964534 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:01.964814 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:01.965086 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] PATH_INFO: `/v3/users/a9aebf7f15f04ad39089fe580c014c88` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:01.966193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9aebf7f15f04ad39089fe580c014c88)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:01.968084 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.014011 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9aebf7f15f04ad39089fe580c014c88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.021246 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9aebf7f15f04ad39089fe580c014c88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.021860 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9aebf7f15f04ad39089fe580c014c88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.028836 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9aebf7f15f04ad39089fe580c014c88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.029472 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9aebf7f15f04ad39089fe580c014c88'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.046840 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1449/2891] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:40:01 2022] DELETE /identity/v3/users/a9aebf7f15f04ad39089fe580c014c88 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:02.051921 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.052456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.052578 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.072732 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:02.073888 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:02.077856 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:02.136653 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1443/2892] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:02.146331 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.147756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:02.153132 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:02.208228 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf189c05-475b-473e-963c-f333efd25bdd admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.211842 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.212474 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.212760 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.213020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.223235 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.224689 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba5a9e-836e-4f98-a521-656819d35f26 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.234037 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1450/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:02.496012 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.497498 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:02.513086 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6bf1-401b-47d7-bbb5-2328700091b7 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.519205 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.520036 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.520443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.520824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] PATH_INFO: `/v3/projects/48392d941b474df7b1b7e26b0c22e069` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.529605 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48392d941b474df7b1b7e26b0c22e069)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.531723 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.584358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48392d941b474df7b1b7e26b0c22e069', 'request_id': 'req-3845e0d4-ef9f-4db8-9546-4510463931b1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.592751 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48392d941b474df7b1b7e26b0c22e069', 'request_id': 'req-3845e0d4-ef9f-4db8-9546-4510463931b1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.598909 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48392d941b474df7b1b7e26b0c22e069', 'request_id': 'req-3845e0d4-ef9f-4db8-9546-4510463931b1'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.610298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] The token cache is being invalidate because project 48392d941b474df7b1b7e26b0c22e069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:40:02.610586 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.611243 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-3845e0d4-ef9f-4db8-9546-4510463931b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:02.613034 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1444/2894] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:40:02 2022] DELETE /identity/v3/projects/48392d941b474df7b1b7e26b0c22e069 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:02.617606 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.617887 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.618157 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.637857 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:02.639048 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:02.700396 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1451/2895] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:02.709808 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.710088 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.710358 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.730212 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:02.731433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:02.739973 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1445/2896] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:02.743999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.754092 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29435-c8c2-48b7-8a57-5e9ea8d1dc1c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.758726 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.759314 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.759593 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.759863 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] PATH_INFO: `/v3/domains` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.760521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:40:02.760820 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.762397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.773696 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1452/2897] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 05:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:40:02.777676 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.786731 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87f7f1-47cb-4617-abb1-60bdf34e1752 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.791362 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.791944 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.792221 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.792525 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.793177 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.795054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.825154 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1446/2898] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 05:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:40:02.829831 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.841683 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a84b9-6271-4e07-93f1-c66432cfcded admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.899200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.899851 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.900063 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.900367 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] PATH_INFO: `/v3/users` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.900992 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.902894 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.923262 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1453/2899] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 05:40:02 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:40:02.928066 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.938178 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b4c40-1d3b-45cc-8fbe-fc34b468d039 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.943417 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.944007 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.944314 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.944591 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.945227 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.946830 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.956619 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1447/2900] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:40:02.960481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:02.970618 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec9d37-d3b1-4efe-8871-37801c145e18 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:02.975526 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:02.976115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:02.976427 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:02.976702 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:02.977332 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:02.978909 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ce09-691c-45af-a91d-b5313700db97 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:02.990105 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1454/2901] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 05:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:40:02.994141 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:03.004680 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92489b-5db8-44dc-94cd-2ab316958e10 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:03.009244 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:03.009881 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:03.010157 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:03.010431 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] PATH_INFO: `/v3/projects/4443d3eb0c4540419fe86007491b169b/users/7d27aeb98d394e50b2ecf45e63b10486/roles/1e1b8d5397754a2e93980b3202ff7e7f` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:03.025737 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4443d3eb0c4540419fe86007491b169b, user_id=7d27aeb98d394e50b2ecf45e63b10486, role_id=1e1b8d5397754a2e93980b3202ff7e7f)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:03.028402 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:03.040761 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1448/2902] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 05:40:02 2022] PUT /identity/v3/projects/4443d3eb0c4540419fe86007491b169b/users/7d27aeb98d394e50b2ecf45e63b10486/roles/1e1b8d5397754a2e93980b3202ff7e7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:40:03.051635 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24e8b-bf9c-4a34-8ee9-f6dea9a2c068 None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:03.051941 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24e8b-bf9c-4a34-8ee9-f6dea9a2c068 None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:03.052214 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24e8b-bf9c-4a34-8ee9-f6dea9a2c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:03.072924 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:03.081112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-a5c24e8b-bf9c-4a34-8ee9-f6dea9a2c068 None None] MFA Rules not processed for user `7d27aeb98d394e50b2ecf45e63b10486`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:03.082548 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a5c24e8b-bf9c-4a34-8ee9-f6dea9a2c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:03.147991 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1455/2903] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:03.159144 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:03.160844 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:03.192013 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:03.217416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5eee-9782-4273-9fe8-5de278fe2fac admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:03.221767 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0102a128d81741c88deeeafedbf0d390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:03.222359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:03.222632 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:03.222898 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:03.230873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:03.232333 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:03.241493 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1449/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:04.606050 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:04.606565 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:04.606686 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:04.626172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] MFA Rules not processed for user `7d27aeb98d394e50b2ecf45e63b10486`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:04.627395 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:04.634200 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1456/2905] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:13.135806 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:13.138172 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:13.263449 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434ae-535d-435a-82d7-becdaf34037b service cinder] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:13.272917 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:13.274051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:13.274051 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:13.274364 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:13.276262 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:13.291083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:13.292829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:13.308295 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1450/2906] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:14.620656 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=30 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:16.407082 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:16.423602 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0806-3159-4082-9aa2-f2871be98f0c None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:16.429532 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:16.430146 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:16.430418 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:16.430877 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:16.432805 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:16.445963 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:16.447467 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:16.460428 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1457/2907] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:16.582784 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:16.584368 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:16.599326 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac86445-69a5-4085-8997-f301e81e6ba4 service nova] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:16.605048 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eadef8d5ac94a3eb3da8d0d3a66d7b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:16.605643 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:16.605922 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:16.606196 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:16.607967 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:16.623323 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:16.625561 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd10a3-445c-411f-91eb-fcb21073c5d1 service nova] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:16.638098 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1451/2908] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:20.791806 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:20.793308 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:20.863572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ffe9a-148e-416c-8f4e-3b7c4e02cef6 service nova] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:20.868310 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:20.869017 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:20.869379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:20.869657 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:20.871397 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:20.947687 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:20.949150 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:20.961563 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1458/2909] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:23.135576 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:23.136805 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:23.137153 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:23.158865 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:23.160774 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:23.222108 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1452/2910] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:40:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:23.250684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:23.252966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:23.339971 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e52ce-db02-4164-a31b-2db1e64383e4 service neutron] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:23.348767 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:23.349388 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:23.350100 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:23.350379 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:23.360530 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:23.363112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885566d-85ec-46f8-a724-00486b984fc6 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:23.372921 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1459/2911] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:40:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:40:31.106373 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:31.107923 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.160119 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=100 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:31.177482 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fe938-d227-41da-87c5-f6a014b68660 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:31.182083 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:31.182713 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:31.182990 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:31.183264 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] PATH_INFO: `/v3/users/12f9f5e7d546480ea24913421273d9b4` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:31.184433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f9f5e7d546480ea24913421273d9b4)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:31.186353 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:31.233572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f9f5e7d546480ea24913421273d9b4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.234155 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f9f5e7d546480ea24913421273d9b4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.242132 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f9f5e7d546480ea24913421273d9b4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.242714 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f9f5e7d546480ea24913421273d9b4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.249559 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f9f5e7d546480ea24913421273d9b4'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.267393 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1453/2912] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:40:31 2022] DELETE /identity/v3/users/12f9f5e7d546480ea24913421273d9b4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:31.435115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:31.435966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:31.436268 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:31.448425 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:31.449829 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.466315 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:31.467523 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.491572 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:31.519897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda77da7-5874-4855-8994-4b5c9843a600 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:31.524553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:31.525182 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:31.525456 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:31.525726 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] PATH_INFO: `/v3/projects/acb97fc9f6ca4cc3970ba2192c2fa3c1` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:31.526841 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb97fc9f6ca4cc3970ba2192c2fa3c1)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:31.528765 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:31.533535 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1460/2913] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:40:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:31.545336 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:31.546681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.580941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb97fc9f6ca4cc3970ba2192c2fa3c1', 'request_id': 'req-d3d0f843-f8b5-4d0e-a619-0126411a2cae'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.588614 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb97fc9f6ca4cc3970ba2192c2fa3c1', 'request_id': 'req-d3d0f843-f8b5-4d0e-a619-0126411a2cae'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.594807 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb97fc9f6ca4cc3970ba2192c2fa3c1', 'request_id': 'req-d3d0f843-f8b5-4d0e-a619-0126411a2cae'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.602782 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:31.610692 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] The token cache is being invalidate because project acb97fc9f6ca4cc3970ba2192c2fa3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:40:31.611477 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.612409 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.614063 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1454/2914] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:40:31 2022] DELETE /identity/v3/projects/acb97fc9f6ca4cc3970ba2192c2fa3c1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:31.617941 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:31.619308 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.630058 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e194fac-002e-45fa-8487-c4e209e64eff None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:31.683052 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f843-f8b5-4d0e-a619-0126411a2cae admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:31.687753 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:31.688433 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:31.688710 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:31.688980 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] PATH_INFO: `/v3/users/94a60b46842f41c1852ea86c196701ca` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:31.690634 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a60b46842f41c1852ea86c196701ca)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:31.692529 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:31.692684 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:31.693135 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:31.693412 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:31.693682 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:31.695964 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:31.740595 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a60b46842f41c1852ea86c196701ca'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.741279 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a60b46842f41c1852ea86c196701ca'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.750062 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a60b46842f41c1852ea86c196701ca'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.750693 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a60b46842f41c1852ea86c196701ca'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.758475 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-4e9ab65c-cbd9-4744-8641-26f111ccdf04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a60b46842f41c1852ea86c196701ca'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:31.762394 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:31.764012 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:31.772907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1461/2915] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:40:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:40:31.776964 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1455/2916] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:40:31 2022] DELETE /identity/v3/users/94a60b46842f41c1852ea86c196701ca => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:40:31.956363 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:31.982425 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e55155-889d-4c60-a770-f01ae849ca59 service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:32.000700 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=100 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:32.050172 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:32.050802 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:32.051079 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:32.051347 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] PATH_INFO: `/v3/projects/16f4765a996048b4930ed42949a4cb58` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:32.052539 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f4765a996048b4930ed42949a4cb58)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:32.054627 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:32.107115 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f4765a996048b4930ed42949a4cb58', 'request_id': 'req-3d15f33f-e0ed-4c31-a60e-156c2dca399c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:32.114451 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f4765a996048b4930ed42949a4cb58', 'request_id': 'req-3d15f33f-e0ed-4c31-a60e-156c2dca399c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:32.125710 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f4765a996048b4930ed42949a4cb58', 'request_id': 'req-3d15f33f-e0ed-4c31-a60e-156c2dca399c'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:32.137324 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] The token cache is being invalidate because project 16f4765a996048b4930ed42949a4cb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:40:32.137684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:32.138564 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:32.140295 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1462/2917] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:40:31 2022] DELETE /identity/v3/projects/16f4765a996048b4930ed42949a4cb58 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:32.331436 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa27b5-d553-4f5e-85c0-869abe57f84d None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:32.331999 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa27b5-d553-4f5e-85c0-869abe57f84d None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:32.332416 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa27b5-d553-4f5e-85c0-869abe57f84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:32.354285 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-90aa27b5-d553-4f5e-85c0-869abe57f84d None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:32.355457 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-90aa27b5-d553-4f5e-85c0-869abe57f84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:32.419733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1456/2918] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:40:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:32.432441 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:32.433922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:32.507208 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15f33f-e0ed-4c31-a60e-156c2dca399c admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:32.512452 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:32.513055 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:32.513329 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:32.513597 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:32.522140 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:32.523717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:32.531481 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1463/2919] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:40:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:40:32.766570 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418102f-65f5-4c6c-adbd-b81321892192 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:32.766848 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418102f-65f5-4c6c-adbd-b81321892192 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:32.767113 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418102f-65f5-4c6c-adbd-b81321892192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:32.787395 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-3418102f-65f5-4c6c-adbd-b81321892192 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:32.788618 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-3418102f-65f5-4c6c-adbd-b81321892192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:32.795637 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1457/2920] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 05:40:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:32.808064 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:32.820681 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706695-a2bf-49a2-8cab-73fffe574827 service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:32.825708 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:32.826303 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:32.826579 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:32.826850 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:32.835790 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:32.837403 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:32.845643 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1464/2921] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:40:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:40:33.157691 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] REQUEST_METHOD: `POST` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:33.157970 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:33.158241 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:33.176873 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.auth.core [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] MFA Rules not processed for user `85e1f5c17df9495182082a9d85fa3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:33.178038 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:33.184968 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1458/2922] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 05:40:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:40:33.197676 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:33.209002 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ca5-de3b-4d45-8fcb-e3c25cf440b6 service swift] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:33.214035 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed5f73bac406439e856d745885c0c092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:33.214592 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:33.214896 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:33.215174 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:33.224073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:33.225717 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffde08-cf8c-4b85-8f9d-610decb673d0 service swift] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:33.234102 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1465/2923] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 05:40:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:40:35.309184 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.310612 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.322293 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:35.379197 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c551178-76ff-4a88-854e-bad638cc1428 None None] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.383548 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.384165 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.384467 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.384736 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] PATH_INFO: `/v3/users/7d27aeb98d394e50b2ecf45e63b10486` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.385846 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d27aeb98d394e50b2ecf45e63b10486)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.387773 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.432907 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d27aeb98d394e50b2ecf45e63b10486'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.433553 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d27aeb98d394e50b2ecf45e63b10486'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.441156 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d27aeb98d394e50b2ecf45e63b10486'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.441908 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d27aeb98d394e50b2ecf45e63b10486'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.449101 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d27aeb98d394e50b2ecf45e63b10486'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.466294 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1459/2924] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:40:35 2022] DELETE /identity/v3/users/7d27aeb98d394e50b2ecf45e63b10486 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:35.470903 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] REQUEST_METHOD: `POST` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.471922 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.472062 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.491782 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.auth.core [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] MFA Rules not processed for user `6c403c92b6314d26999d2aed9104bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:35.493073 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.496920 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:35.559938 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1466/2925] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 05:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:35.569716 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.571146 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.630862 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae4271-81c4-4161-aedd-0d66ceeab163 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.634375 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.634977 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.635426 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.635878 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.649572 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.650993 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.661010 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1460/2926] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:35.828556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.829994 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.842341 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cacfa-54d0-419a-9ce0-49373956d2da None None] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.847584 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.848476 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.848750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.849020 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] PATH_INFO: `/v3/projects/4443d3eb0c4540419fe86007491b169b` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.850152 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4443d3eb0c4540419fe86007491b169b)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.852041 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.901287 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4443d3eb0c4540419fe86007491b169b', 'request_id': 'req-9fef3369-4b48-4460-9f41-193f0763a4a2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.907721 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4443d3eb0c4540419fe86007491b169b', 'request_id': 'req-9fef3369-4b48-4460-9f41-193f0763a4a2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.915947 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4443d3eb0c4540419fe86007491b169b', 'request_id': 'req-9fef3369-4b48-4460-9f41-193f0763a4a2'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.926684 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] The token cache is being invalidate because project 4443d3eb0c4540419fe86007491b169b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:40:35.926965 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.927586 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.929087 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1467/2927] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:40:35 2022] DELETE /identity/v3/projects/4443d3eb0c4540419fe86007491b169b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:45.656135 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:45.923372 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:07.911065 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:07.920489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4ca3-b6b1-4fa4-9d79-398dd4770aef service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:07.972298 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a938ac6d7d3b42ab9e8aa8b2b9ea9410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:07.972900 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:07.973173 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:07.973441 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:07.975166 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:08.036712 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:08.038131 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:08.048269 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1461/2928] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:11.067229 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:11.068748 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:11.172915 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef3369-4b48-4460-9f41-193f0763a4a2 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:11.178193 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:11.178892 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:11.179240 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:11.179591 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:11.181750 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:11.194556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:11.197129 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:11.207773 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1468/2929] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:14.283527 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:14.285022 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:14.297617 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c0854-3fc8-4160-9890-76ff25b3a24f service neutron] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:14.302194 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:14.302756 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:14.303030 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:14.303299 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] PATH_INFO: `/v3/users/57daecc890ac421b8eb7b5fb4bf25411` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:14.304443 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57daecc890ac421b8eb7b5fb4bf25411)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:14.306855 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:14.352839 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57daecc890ac421b8eb7b5fb4bf25411'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.353489 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57daecc890ac421b8eb7b5fb4bf25411'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.360875 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57daecc890ac421b8eb7b5fb4bf25411'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.361531 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57daecc890ac421b8eb7b5fb4bf25411'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.369114 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57daecc890ac421b8eb7b5fb4bf25411'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.386309 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1462/2930] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:42:14 2022] DELETE /identity/v3/users/57daecc890ac421b8eb7b5fb4bf25411 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:42:14.564588 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:14.577014 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590d14c-4533-45ce-80d3-2bf4d006d85c service placement] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:14.634521 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:14.635141 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:14.635414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:14.635699 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] PATH_INFO: `/v3/projects/774c63ee5fc74c619f63c64590c154e4` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:14.636811 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774c63ee5fc74c619f63c64590c154e4)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:14.638678 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:14.682810 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774c63ee5fc74c619f63c64590c154e4', 'request_id': 'req-6d13bb0b-4844-4e13-8084-48509e24c114'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.688954 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774c63ee5fc74c619f63c64590c154e4', 'request_id': 'req-6d13bb0b-4844-4e13-8084-48509e24c114'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.696662 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774c63ee5fc74c619f63c64590c154e4', 'request_id': 'req-6d13bb0b-4844-4e13-8084-48509e24c114'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.706756 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] The token cache is being invalidate because project 774c63ee5fc74c619f63c64590c154e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:42:14.707038 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.707655 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.709170 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1469/2931] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:42:14 2022] DELETE /identity/v3/projects/774c63ee5fc74c619f63c64590c154e4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:42:14.713843 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:14.715359 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:14.728824 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a85ec-8780-46b7-ac85-6f98619c2185 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:14.746349 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:14.785813 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:14.786361 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:14.786635 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:14.786939 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] PATH_INFO: `/v3/users/a9c5d43e80d742ff9f750b057dcd11a5` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:14.788054 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c5d43e80d742ff9f750b057dcd11a5)` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:14.790078 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:14.834759 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c5d43e80d742ff9f750b057dcd11a5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.835397 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c5d43e80d742ff9f750b057dcd11a5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.843439 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c5d43e80d742ff9f750b057dcd11a5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.844104 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c5d43e80d742ff9f750b057dcd11a5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.851058 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.notifications [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c5d43e80d742ff9f750b057dcd11a5'} {{(pid=81542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:14.868897 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1463/2932] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 05:42:14 2022] DELETE /identity/v3/users/a9c5d43e80d742ff9f750b057dcd11a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:42:15.040507 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:15.041966 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:15.055843 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13bb0b-4844-4e13-8084-48509e24c114 admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:15.074112 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:15.117257 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c403c92b6314d26999d2aed9104bf3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3af173ac0b7d40069de1fa09770699a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:15.118414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:15.118691 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:15.118962 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] PATH_INFO: `/v3/projects/297020a5021c4784889de1ea9e461280` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:15.120148 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=297020a5021c4784889de1ea9e461280)` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:15.122070 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:15.168006 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297020a5021c4784889de1ea9e461280', 'request_id': 'req-a21f6e1c-1409-40fd-a479-42c61a73c30f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:15.174265 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297020a5021c4784889de1ea9e461280', 'request_id': 'req-a21f6e1c-1409-40fd-a479-42c61a73c30f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:15.183630 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297020a5021c4784889de1ea9e461280', 'request_id': 'req-a21f6e1c-1409-40fd-a479-42c61a73c30f'} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:15.194967 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] The token cache is being invalidate because project 297020a5021c4784889de1ea9e461280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:42:15.195249 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:15.196519 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.notifications [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:42:15.198094 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1470/2933] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 05:42:15 2022] DELETE /identity/v3/projects/297020a5021c4784889de1ea9e461280 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:42:24.864546 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:24.991953 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Authenticating user token {{(pid=81542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:24.994023 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:25.065557 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce7f5-7437-4c36-9d4e-b03e776b09b9 admin admin] Validating token access rules against request {{(pid=81542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:25.125556 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:25.126122 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] REQUEST_METHOD: `GET` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:25.126419 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] SCRIPT_NAME: `/identity` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:25.126800 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:25.128577 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.fernet_utils [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:25.139733 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:25.141332 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d93726-9d3e-45be-9864-81e507ec0305 service placement] RBAC: Authorization granted {{(pid=81542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:25.151193 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: [pid: 81542|app: 0|req: 1464/2934] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:25.192034 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:35.146948 dv-cinder-859186-2-iscsi devstack@keystone.service[81542]: DEBUG dbcounter [-] [81542] Writing DB stats keystone:SELECT=58 {{(pid=81542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:29.987414 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Authenticating user token {{(pid=81543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:29.988907 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.fernet_utils [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:30.072999 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f6e1c-1409-40fd-a479-42c61a73c30f admin admin] Validating token access rules against request {{(pid=81543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:30.078061 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35bda10601d04f22a509cfb45e19c6ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f32b9319d9146eb847509775ea6af53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:30.079196 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] REQUEST_METHOD: `GET` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.079556 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] SCRIPT_NAME: `/identity` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.079830 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.088646 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:30.090203 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea76a-6aa9-4963-9843-43d9b3a1d328 service placement] RBAC: Authorization granted {{(pid=81543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:30.100665 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: [pid: 81543|app: 0|req: 1471/2935] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 05:47:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:47:40.096389 dv-cinder-859186-2-iscsi devstack@keystone.service[81543]: DEBUG dbcounter [-] [81543] Writing DB stats keystone:SELECT=37 {{(pid=81543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}